[llvm] [NVVM][NVPTX] Support spcompress and spdecompress intrinsics (PR #221751)
Kirill Vedernikov via llvm-commits
llvm-commits at lists.llvm.org
Sun Sep 20 22:01:55 PDT 2026
https://github.com/kvederni updated https://github.com/llvm/llvm-project/pull/221751
>From fe997a0a3543865338aecd1b719a2b9f58ab0493 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Mon, 7 Sep 2026 16:43:04 +0200
Subject: [PATCH 01/10] [NVVM][NVPTX] Support spcompress and spdecompress
intrinsics
---
llvm/docs/NVPTXUsage.md | 108 +++
llvm/include/llvm/IR/IntrinsicsNVVM.td | 59 ++
llvm/include/llvm/IR/NVVMIntrinsicUtils.h | 132 +++
llvm/lib/IR/NVVMIntrinsicUtils.cpp | 17 +-
.../NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp | 111 +++
.../NVPTX/MCTargetDesc/NVPTXInstPrinter.h | 2 +
llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp | 199 ++++
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp | 122 +++
llvm/lib/Target/NVPTX/NVPTXInstrInfo.td | 3 +
llvm/lib/Target/NVPTX/NVPTXIntrinsics.td | 70 ++
llvm/test/CodeGen/NVPTX/spcompress-invalid.ll | 83 ++
llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll | 860 ++++++++++++++++++
.../CodeGen/NVPTX/spdecompress-invalid.ll | 105 +++
.../CodeGen/NVPTX/spdecompress-sp1to16.ll | 309 +++++++
.../test/CodeGen/NVPTX/spdecompress-sp1to2.ll | 552 +++++++++++
.../test/CodeGen/NVPTX/spdecompress-sp1to4.ll | 682 ++++++++++++++
.../test/CodeGen/NVPTX/spdecompress-sp1to8.ll | 350 +++++++
.../CodeGen/NVPTX/spdecompress-sp2to16.ll | 325 +++++++
.../test/CodeGen/NVPTX/spdecompress-sp2to4.ll | 758 +++++++++++++++
.../test/CodeGen/NVPTX/spdecompress-sp2to8.ll | 379 ++++++++
.../CodeGen/NVPTX/spdecompress-sp4to16.ll | 184 ++++
.../test/CodeGen/NVPTX/spdecompress-sp4to8.ll | 224 +++++
22 files changed, 5633 insertions(+), 1 deletion(-)
create mode 100644 llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
diff --git a/llvm/docs/NVPTXUsage.md b/llvm/docs/NVPTXUsage.md
index 67bf8ea482933a..a048ed047c46cc 100644
--- a/llvm/docs/NVPTXUsage.md
+++ b/llvm/docs/NVPTXUsage.md
@@ -1028,6 +1028,114 @@ If the given pointer in the generic address space refers to memory which falls
within the state space of the intrinsic (and therefore could be safely address
space casted to this space), 1 is returned, otherwise 0 is returned.
+### Structured Sparsity Intrinsics
+
+The `llvm.nvvm.spcompress.*` and `llvm.nvvm.spdecompress.*` intrinsics model
+the PTX `spcompress` and `spdecompress` instructions. They require PTX ISA 9.4
+and `sm_107a`.
+
+The intrinsic types representing `mdata`, `cdata`, and `data` are overloaded.
+A bundle of one 32-bit register is represented by `i32`, and a bundle of `N`
+registers is represented by `<N x i32>`. The actual intrinsic names include
+the corresponding LLVM overload suffixes.
+
+The three trailing arguments are immediate qualifiers with the following
+encodings:
+
+| Argument | Values | PTX qualifiers |
+| ----------------------- | ------------------------- | -------------------------------------- |
+| `%elem_size` | `8`, `16` | `.b8`, `.b16` |
+| `%idx_size` | `2`, `4` | `.b2`, `.b4` |
+| `%repeat_factor` | `0`, `1`, ..., `6` | `.x1`, `.x2`, ..., `.x64` |
+
+The repeat factor `num` is `1 << %repeat_factor`.
+
+#### '`llvm.nvvm.spcompress.sp2to4`' Intrinsic
+
+##### Syntax:
+
+```llvm
+declare {MDataTy, CDataTy} @llvm.nvvm.spcompress.sp2to4(
+ DataTy %data, i32 %spdesc, i32 immarg %elem_size,
+ i32 immarg %idx_size, i32 immarg %repeat_factor)
+```
+
+##### Overview:
+
+This intrinsic compresses the dense vector `%data` using 2:4 structured
+sparsity. It returns the selected element indices as `mdata` and the selected
+elements as `cdata`. The number of 32-bit registers in each bundle is:
+
+| Bundle | Register count |
+| ------- | ------------------------------------------- |
+| `data` | `2 * num` |
+| `cdata` | `num` |
+| `mdata` | `ceil(num * %idx_size / %elem_size)` |
+
+The combined `mdata`, `cdata`, and `data` bundle size must not exceed 253
+registers.
+
+The `%spdesc` operand specifies the selection operation and the element data
+type:
+
+| Bits | Description | Values |
+| ---- | ------------------- | ------------------------------------------------------------------------- |
+| 0-1 | Selection operation | `0`: max, `1`: maxabs, `2`: min, `3`: minabs |
+| 2-4 | Element data type | `0`: f16/u8, `1`: bf16/s8, `2`: e5m2, `3`: e4m3, `4`: e3m2, `5`: e2m3 |
+| 5-31 | Reserved | `0` |
+
+The element data type must be consistent with `%elem_size`. When the metadata
+occupies less than 32 bits, it is zero-extended to fill its `i32` register.
+The operation treats negative zero as less than positive zero. NaN elements
+are always selected; when multiple selections satisfy the same comparison,
+the selected indices are implementation-specific.
+
+For more information, see the
+[PTX ISA](https://docs.nvidia.com/cuda/parallel-thread-execution/#data-movement-and-conversion-instructions-spcompress).
+
+#### '`llvm.nvvm.spdecompress.*`' Intrinsics
+
+##### Syntax:
+
+```llvm
+declare DataTy @llvm.nvvm.spdecompress.{sp1to2,sp1to4,sp1to8,sp1to16,
+ sp2to4,sp2to8,sp2to16,
+ sp4to8,sp4to16}(
+ MDataTy %mdata, CDataTy %cdata, i32 immarg %elem_size,
+ i32 immarg %idx_size, i32 immarg %repeat_factor)
+```
+
+##### Overview:
+
+These intrinsics decompress the structured sparse vector `%cdata` into a
+dense `data` vector. The intrinsic name specifies the number of source and
+target elements in each group. For example, `sp2to4` maps two compressed
+elements into a group of four dense elements using the indices in `%mdata`.
+Dense positions not selected by the metadata are set to zero.
+
+For an intrinsic named `spXtoY`, the number of 32-bit registers in each bundle
+is:
+
+| Bundle | Register count |
+| ------- | --------------------------------------------------- |
+| `mdata` | `ceil(X * %idx_size * num / 32)` |
+| `cdata` | `ceil(X * %elem_size * num / 32)` |
+| `data` | `ceil(Y * %elem_size * num / 32)` |
+
+The following conditions must hold:
+
+- `X * %elem_size <= 32`.
+- `%idx_size` is `2` only when `Y <= 4`.
+- `32 <= Y * %elem_size * num <= 4096`.
+- The combined `mdata`, `cdata`, and `data` bundle size does not exceed 253
+ registers.
+
+The behavior is implementation-specific if a metadata index does not identify
+a position in its target group.
+
+For more information, see the
+[PTX ISA](https://docs.nvidia.com/cuda/parallel-thread-execution/#data-movement-and-conversion-instructions-spdecompress).
+
### Narrow Floating-Point Conversion intrinsics
These intrinsics perform conversions involving narrow floating-point formats.
diff --git a/llvm/include/llvm/IR/IntrinsicsNVVM.td b/llvm/include/llvm/IR/IntrinsicsNVVM.td
index ff12175180d149..d3dc735785552f 100644
--- a/llvm/include/llvm/IR/IntrinsicsNVVM.td
+++ b/llvm/include/llvm/IR/IntrinsicsNVVM.td
@@ -145,6 +145,10 @@ def llvm_local_ptr_ty : LLVMQualPointerType<5>; // (local)ptr
def llvm_tmem_ptr_ty : LLVMQualPointerType<6>; // (tensor memory)ptr
def llvm_shared_cluster_ptr_ty : LLVMQualPointerType<7>; // (shared_cluster)ptr
+// A RangeList is a RangeSet with singleton closed ranges for each listed value.
+class RangeList<ArgIndex idx, list<int> values>
+ : RangeSet<idx, !foreach(value, values, [value, value])>;
+
//
// MISC
//
@@ -1331,6 +1335,23 @@ class NVVMPureIntrinsic<list<LLVMType> ret_types,
DefaultAttrsIntrinsic<ret_types, param_types,
intr_properties # [IntrNoMem, IntrSpeculatable], name> {}
+defvar NVVM_SP_LG2_REPEAT_FACTOR_LIMIT = 7;
+
+class NVVM_SP_ARGPROPS<ArgIndex ElemSizeIdx, ArgIndex IdxSizeIdx,
+ ArgIndex RepeatFactorIdx> {
+ list<IntrinsicProperty> Prop = [
+ RangeList<ElemSizeIdx, [8, 16]>,
+ RangeList<IdxSizeIdx, [2, 4]>,
+ Range<RepeatFactorIdx, 0, NVVM_SP_LG2_REPEAT_FACTOR_LIMIT>,
+ ArgInfo<ElemSizeIdx, [ArgName<"elem_size">,
+ ImmArgPrinter<"printSPElemSize">]>,
+ ArgInfo<IdxSizeIdx, [ArgName<"idx_size">,
+ ImmArgPrinter<"printSPIdxSize">]>,
+ ArgInfo<RepeatFactorIdx, [ArgName<"repeat_factor">,
+ ImmArgPrinter<"printSPRepeatFactor">]>
+ ];
+}
+
// For im2col mode the number of offsets is (dim - 2).
// For im2col_w / im2col_w_128 modes it is always 2 (wHalo and wOffset),
// irrespective of dim.
@@ -4214,4 +4235,42 @@ def int_nvvm_tensormap_replace_fill_mode :
ArgInfo<ArgIndex<1>, [ArgName<"fill_mode">,
ImmArgPrinter<"printTensormapFillMode">]>]>;
+//
+// spcompress intrinsics
+//
+
+class SPCompress :
+ DefaultAttrsIntrinsicFlags<
+ [llvm_anyint_ty, // mdata
+ llvm_anyint_ty], // cdata
+ [llvm_anyint_ty, // data
+ llvm_i32_ty], // spdesc
+ [llvm_i32_ty, // elem_size
+ llvm_i32_ty, // idx_size
+ llvm_i32_ty], // repeat_factor: 0..6 for .x1 .. .x64
+ !listconcat([IntrNoMem, IntrSpeculatable],
+ NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>, ArgIndex<4>>.Prop)>;
+
+def int_nvvm_spcompress_sp2to4 : SPCompress;
+
+//
+// spdecompress intrinsics
+//
+
+class SPDecompress :
+ DefaultAttrsIntrinsicFlags<
+ [llvm_anyint_ty], // data
+ [llvm_anyint_ty, // mdata
+ llvm_anyint_ty], // cdata
+ [llvm_i32_ty, // elem_size
+ llvm_i32_ty, // idx_size
+ llvm_i32_ty], // repeat_factor: 0..6 for .x1 .. .x64
+ !listconcat([IntrNoMem, IntrSpeculatable],
+ NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>, ArgIndex<4>>.Prop)>;
+
+foreach num_src = [1, 2, 4] in
+ foreach num_tgt = [2, 4, 8, 16] in
+ if !lt(num_src, num_tgt) then
+ def int_nvvm_spdecompress_sp # num_src # to # num_tgt : SPDecompress;
+
} // let TargetPrefix = "nvvm"
diff --git a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
index 9585cdac2a4ea9..f1562fe948cd43 100644
--- a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
+++ b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
@@ -19,6 +19,7 @@
#include "llvm/ADT/APFloat.h"
#include "llvm/ADT/APInt.h"
+#include "llvm/ADT/Enum.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/IR/Constants.h"
#include "llvm/IR/Intrinsics.h"
@@ -215,6 +216,133 @@ enum class MBarrierLayout : uint8_t {
V1 = 1,
};
+enum class SPElemSize : uint64_t {
+ B8 = 8,
+ B16 = 16,
+};
+
+enum class SPIdxSize : uint64_t {
+ B2 = 2,
+ B4 = 4,
+};
+
+enum class SPLg2RepeatFactor : uint64_t {
+ X1 = 0,
+ X2 = 1,
+ X4 = 2,
+ X8 = 3,
+ X16 = 4,
+ X32 = 5,
+ X64 = 6,
+};
+
+struct SPOperandLayout {
+ unsigned MetadataSize;
+ unsigned CompressedDataSize;
+ unsigned DataSize;
+};
+
+// PTX limits the combined vector size of the mdata, cdata, and data operands
+// of spcompress and spdecompress to 253 32-bit registers.
+inline constexpr unsigned MaxSPOperandRegisters = 253;
+
+inline StringRef getSPElemSizeName(SPElemSize ElemSize) {
+ static constexpr EnumStringDef<SPElemSize> SPElemSizeNameDefs[] = {
+ {{".b8"}, SPElemSize::B8},
+ {{".b16"}, SPElemSize::B16},
+ };
+ static constexpr auto SPElemSizeNames =
+ BUILD_ENUM_STRINGS(SPElemSizeNameDefs);
+ return EnumStrings(SPElemSizeNames).toString(ElemSize);
+}
+
+inline StringRef getSPIdxSizeName(SPIdxSize IdxSize) {
+ static constexpr EnumStringDef<SPIdxSize> SPIdxSizeNameDefs[] = {
+ {{".b2"}, SPIdxSize::B2},
+ {{".b4"}, SPIdxSize::B4},
+ };
+ static constexpr auto SPIdxSizeNames = BUILD_ENUM_STRINGS(SPIdxSizeNameDefs);
+ return EnumStrings(SPIdxSizeNames).toString(IdxSize);
+}
+
+inline StringRef getSPRepeatFactorName(SPLg2RepeatFactor Lg2RepeatFactor) {
+ static constexpr EnumStringDef<SPLg2RepeatFactor> SPRepeatFactorNameDefs[] = {
+ {{".x1"}, SPLg2RepeatFactor::X1}, {{".x2"}, SPLg2RepeatFactor::X2},
+ {{".x4"}, SPLg2RepeatFactor::X4}, {{".x8"}, SPLg2RepeatFactor::X8},
+ {{".x16"}, SPLg2RepeatFactor::X16}, {{".x32"}, SPLg2RepeatFactor::X32},
+ {{".x64"}, SPLg2RepeatFactor::X64},
+ };
+ static constexpr auto SPRepeatFactorNames =
+ BUILD_ENUM_STRINGS(SPRepeatFactorNameDefs);
+ return EnumStrings(SPRepeatFactorNames).toString(Lg2RepeatFactor);
+}
+
+inline unsigned getSPRepeatFactor(unsigned Lg2RepeatFactor) {
+ return 1U << Lg2RepeatFactor;
+}
+
+inline bool isValidSPDecompressFactor(unsigned NumSrc, unsigned NumTgt) {
+ switch (NumSrc) {
+ case 1:
+ return NumTgt == 2 || NumTgt == 4 || NumTgt == 8 || NumTgt == 16;
+ case 2:
+ return NumTgt == 4 || NumTgt == 8 || NumTgt == 16;
+ case 4:
+ return NumTgt == 8 || NumTgt == 16;
+ default:
+ return false;
+ }
+}
+
+inline std::optional<SPOperandLayout>
+getSPCompressLayout(unsigned ElemSize, unsigned IdxSize,
+ unsigned Lg2RepeatFactor) {
+ if (getSPElemSizeName(static_cast<SPElemSize>(ElemSize)).empty() ||
+ getSPIdxSizeName(static_cast<SPIdxSize>(IdxSize)).empty() ||
+ getSPRepeatFactorName(static_cast<SPLg2RepeatFactor>(Lg2RepeatFactor))
+ .empty())
+ return std::nullopt;
+
+ unsigned RepeatFactor = getSPRepeatFactor(Lg2RepeatFactor);
+ SPOperandLayout Layout = {
+ divideCeil(RepeatFactor * IdxSize, ElemSize),
+ RepeatFactor,
+ RepeatFactor * 2,
+ };
+ if (Layout.MetadataSize + Layout.CompressedDataSize + Layout.DataSize >
+ MaxSPOperandRegisters)
+ return std::nullopt;
+ return Layout;
+}
+
+inline std::optional<SPOperandLayout>
+getSPDecompressLayout(unsigned NumSrc, unsigned NumTgt, unsigned ElemSize,
+ unsigned IdxSize, unsigned Lg2RepeatFactor) {
+ if (!isValidSPDecompressFactor(NumSrc, NumTgt) ||
+ getSPElemSizeName(static_cast<SPElemSize>(ElemSize)).empty() ||
+ getSPIdxSizeName(static_cast<SPIdxSize>(IdxSize)).empty() ||
+ getSPRepeatFactorName(static_cast<SPLg2RepeatFactor>(Lg2RepeatFactor))
+ .empty())
+ return std::nullopt;
+ if (NumSrc * ElemSize > 32 || (IdxSize == 2 && NumTgt > 4))
+ return std::nullopt;
+
+ unsigned RepeatFactor = getSPRepeatFactor(Lg2RepeatFactor);
+ unsigned DataBits = NumTgt * ElemSize * RepeatFactor;
+ if (DataBits < 32 || DataBits > 4096)
+ return std::nullopt;
+
+ SPOperandLayout Layout = {
+ divideCeil(NumSrc * IdxSize * RepeatFactor, 32),
+ divideCeil(NumSrc * ElemSize * RepeatFactor, 32),
+ divideCeil(DataBits, 32),
+ };
+ if (Layout.MetadataSize + Layout.CompressedDataSize + Layout.DataSize >
+ MaxSPOperandRegisters)
+ return std::nullopt;
+ return Layout;
+}
+
LLVM_ABI void printTcgen05MMAKind(raw_ostream &OS, const Constant *ImmArgVal);
LLVM_ABI void printMBarrierLayout(raw_ostream &OS, const Constant *ImmArgVal);
@@ -234,6 +362,10 @@ LLVM_ABI void printTcgen05CollectorUsageOp(raw_ostream &OS,
LLVM_ABI void printTcgen05MMACollectorBBuffer(raw_ostream &OS,
const Constant *ImmArgVal);
+LLVM_ABI void printSPElemSize(raw_ostream &OS, const Constant *ImmArgVal);
+LLVM_ABI void printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal);
+LLVM_ABI void printSPRepeatFactor(raw_ostream &OS, const Constant *ImmArgVal);
+
LLVM_ABI void printTensormapElemType(raw_ostream &OS,
const Constant *ImmArgVal);
LLVM_ABI void printTensormapInterleaveLayout(raw_ostream &OS,
diff --git a/llvm/lib/IR/NVVMIntrinsicUtils.cpp b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
index e5a45fd5e9cd9b..5e176da6878f76 100644
--- a/llvm/lib/IR/NVVMIntrinsicUtils.cpp
+++ b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
@@ -11,7 +11,6 @@
//===----------------------------------------------------------------------===//
#include "llvm/IR/NVVMIntrinsicUtils.h"
-#include "llvm/ADT/StringRef.h"
using namespace llvm;
using namespace nvvm;
@@ -137,6 +136,22 @@ void nvvm::printTcgen05MMACollectorBBuffer(raw_ostream &OS,
}
}
+void nvvm::printSPElemSize(raw_ostream &OS, const Constant *ImmArgVal) {
+ if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
+ OS << getSPElemSizeName(static_cast<SPElemSize>(CI->getZExtValue()));
+}
+
+void nvvm::printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal) {
+ if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
+ OS << getSPIdxSizeName(static_cast<SPIdxSize>(CI->getZExtValue()));
+}
+
+void nvvm::printSPRepeatFactor(raw_ostream &OS, const Constant *ImmArgVal) {
+ if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
+ OS << getSPRepeatFactorName(
+ static_cast<SPLg2RepeatFactor>(CI->getZExtValue()));
+}
+
void nvvm::printTensormapElemType(raw_ostream &OS, const Constant *ImmArgVal) {
static constexpr StringRef TensormapElemTypes[] = {
"u8", "u16", "u32", "s32", "u64", "s64",
diff --git a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
index dc250c67c3bc91..44992c36d3930d 100644
--- a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
+++ b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
@@ -474,6 +474,117 @@ void NVPTXInstPrinter::printMmaCode(const MCInst *MI, int OpNum,
llvm_unreachable("Unknown Modifier");
}
+static void printSPRegList(NVPTXInstPrinter *Printer, const MCInst *MI,
+ unsigned FirstOp, unsigned NumOps,
+ const MCSubtargetInfo &STI, raw_ostream &O) {
+ O << "{";
+ for (unsigned I = 0; I != NumOps; ++I) {
+ if (I != 0)
+ O << ", ";
+ Printer->printOperand(MI, FirstOp + I, STI, O);
+ }
+ O << "}";
+}
+
+void NVPTXInstPrinter::printSPQualifier(const MCInst *MI, int OpNum,
+ const MCSubtargetInfo &STI,
+ raw_ostream &O, StringRef Modifier) {
+ if (Modifier.empty()) {
+ printOperand(MI, OpNum, STI, O);
+ return;
+ }
+
+ unsigned Imm = MI->getOperand(OpNum).getImm();
+ if (Modifier == "elem_size") {
+ StringRef ElemSize =
+ nvvm::getSPElemSizeName(static_cast<nvvm::SPElemSize>(Imm));
+ assert(!ElemSize.empty() && "invalid SP elem_size");
+ O << ElemSize;
+ return;
+ }
+
+ if (Modifier == "idx_size") {
+ StringRef IdxSize =
+ nvvm::getSPIdxSizeName(static_cast<nvvm::SPIdxSize>(Imm));
+ assert(!IdxSize.empty() && "invalid SP idx_size");
+ O << IdxSize;
+ return;
+ }
+
+ if (Modifier == "repeat_factor") {
+ StringRef RepeatFactor =
+ nvvm::getSPRepeatFactorName(static_cast<nvvm::SPLg2RepeatFactor>(Imm));
+ assert(!RepeatFactor.empty() && "invalid SP repeat_factor");
+ O << RepeatFactor;
+ return;
+ }
+
+ if (Modifier == "spfactor") {
+ unsigned NumTgt = MI->getOperand(OpNum + 1).getImm();
+ assert(nvvm::isValidSPDecompressFactor(Imm, NumTgt) && "invalid spfactor");
+ O << ".sp::" << Imm << ":" << NumTgt;
+ return;
+ }
+
+ auto GetSPLayout = [&](auto GetLayout) {
+ return GetLayout(Imm,
+ static_cast<unsigned>(MI->getOperand(OpNum + 1).getImm()),
+ static_cast<unsigned>(MI->getOperand(OpNum + 2).getImm()));
+ };
+
+ auto PrintSPRegLists = [&](unsigned FirstOp0, unsigned NumOps0,
+ unsigned FirstOp1, unsigned NumOps1,
+ unsigned FirstOp2, unsigned NumOps2) {
+ printSPRegList(this, MI, FirstOp0, NumOps0, STI, O);
+ O << ", ";
+ printSPRegList(this, MI, FirstOp1, NumOps1, STI, O);
+ O << ", ";
+ printSPRegList(this, MI, FirstOp2, NumOps2, STI, O);
+ };
+
+ if (Modifier == "spcompress_ops") {
+ auto Layout = GetSPLayout(nvvm::getSPCompressLayout);
+ assert(Layout && "invalid spcompress qualifiers");
+
+ unsigned FirstVariableOp = OpNum + 3;
+ assert(static_cast<unsigned>(OpNum) ==
+ Layout->MetadataSize + Layout->CompressedDataSize &&
+ MI->getNumOperands() == FirstVariableOp + Layout->DataSize + 1 &&
+ "invalid spcompress operand layout");
+
+ PrintSPRegLists(0, Layout->MetadataSize, Layout->MetadataSize,
+ Layout->CompressedDataSize, FirstVariableOp,
+ Layout->DataSize);
+ O << ", ";
+ printOperand(MI, FirstVariableOp + Layout->DataSize, STI, O);
+ return;
+ }
+
+ if (Modifier == "spdecompress_ops") {
+ unsigned NumSrc = MI->getOperand(OpNum + 3).getImm();
+ unsigned NumTgt = MI->getOperand(OpNum + 4).getImm();
+ auto Layout = GetSPLayout(
+ [&](unsigned ElemSize, unsigned IdxSize, unsigned Lg2RepeatFactor) {
+ return nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSize, IdxSize,
+ Lg2RepeatFactor);
+ });
+ assert(Layout && "invalid spdecompress qualifiers");
+
+ unsigned FirstVariableOp = OpNum + 5;
+ assert(static_cast<unsigned>(OpNum) == Layout->DataSize &&
+ MI->getNumOperands() == FirstVariableOp + Layout->MetadataSize +
+ Layout->CompressedDataSize &&
+ "invalid spdecompress operand layout");
+
+ PrintSPRegLists(0, Layout->DataSize, FirstVariableOp, Layout->MetadataSize,
+ FirstVariableOp + Layout->MetadataSize,
+ Layout->CompressedDataSize);
+ return;
+ }
+
+ llvm_unreachable(formatv("Unknown Modifier: {}", Modifier).str().c_str());
+}
+
void NVPTXInstPrinter::printMemOperand(const MCInst *MI, int OpNum,
const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier) {
diff --git a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
index dc3e229364d0e8..dd8e7b499e3f9e 100644
--- a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
+++ b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
@@ -53,6 +53,8 @@ class NVPTXInstPrinter : public MCInstPrinter {
raw_ostream &O);
void printMmaCode(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier = {});
+ void printSPQualifier(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
+ raw_ostream &O, StringRef Modifier = {});
void printMemOperand(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier = {});
void printUsedBytesMaskPragma(const MCInst *MI, int OpNum,
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
index 81badbc1c3b101..39196bffd851d8 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
@@ -115,6 +115,7 @@ class NVPTXDAGToDAGISel : public SelectionDAGISel {
#include "NVPTXGenDAGISel.inc"
void Select(SDNode *N) override;
+ bool tryIntrinsicNoChain(SDNode *N);
bool tryIntrinsicChain(SDNode *N);
bool tryIntrinsicVoid(SDNode *N);
void SelectTexSurfHandle(SDNode *N);
@@ -138,6 +139,8 @@ class NVPTXDAGToDAGISel : public SelectionDAGISel {
bool IsIm2Col = false);
void SelectTcgen05Ld(SDNode *N, bool hasOffset = false);
void SelectTcgen05St(SDNode *N, bool hasOffset = false);
+ void selectSPCompress(SDNode *N);
+ void selectSPDecompress(SDNode *N, unsigned NumSrc, unsigned NumTgt);
void selectAtomicSwap128(SDNode *N);
inline SDValue getI32Imm(unsigned Imm, const SDLoc &DL) {
@@ -289,6 +292,10 @@ void NVPTXDAGToDAGISel::Select(SDNode *N) {
if (tryStoreVector(N))
return;
break;
+ case ISD::INTRINSIC_WO_CHAIN:
+ if (tryIntrinsicNoChain(N))
+ return;
+ break;
case ISD::INTRINSIC_W_CHAIN:
if (tryIntrinsicChain(N))
return;
@@ -441,6 +448,198 @@ void NVPTXDAGToDAGISel::SelectTcgen05Ld(SDNode *N, bool hasOffset) {
}
}
+static void checkSPRegTypes(SDNode *N, unsigned NumFlags, StringRef Name) {
+ for (const EVT VT : N->values())
+ if (VT != MVT::i32)
+ report_fatal_error(Twine(Name) + " expects scalarized i32 results");
+
+ for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
+ if (N->getOperand(I).getValueType() != MVT::i32)
+ report_fatal_error(Twine(Name) + " expects scalarized i32 operands");
+}
+
+// Keep these lists in sync with their counterparts in NVPTXIntrinsics.td. A
+// C++-only count references a nonexistent opcode and fails to compile; a
+// TableGen-only count reaches llvm_unreachable in the selectors below.
+#define SPCOMPRESS_OPCODE_CASES(CASE_MACRO) \
+ /* .b8.b2 and .b16.b4: ceil(repeat_factor / 4) + repeat_factor. */ \
+ /* The repeat_factor operand encodes the exponent. */ \
+ CASE_MACRO(2) /* .x1 */ \
+ CASE_MACRO(3) /* .x2 */ \
+ CASE_MACRO(5) /* .x4 */ \
+ CASE_MACRO(10) /* .x8 */ \
+ CASE_MACRO(20) /* .x16 */ \
+ CASE_MACRO(40) /* .x32 */ \
+ CASE_MACRO(80) /* .x64 */ \
+ /* .b16.b2: additional unique counts after the duplicate small cases. */ \
+ CASE_MACRO(9) /* .x8 */ \
+ CASE_MACRO(18) /* .x16 */ \
+ CASE_MACRO(36) /* .x32 */ \
+ CASE_MACRO(72) /* .x64 */ \
+ /* .b8.b4: additional unique counts after the duplicate small cases. */ \
+ CASE_MACRO(6) /* .x4 */ \
+ CASE_MACRO(12) /* .x8 */ \
+ CASE_MACRO(24) /* .x16 */ \
+ CASE_MACRO(48) /* .x32 */ \
+ CASE_MACRO(96) /* .x64 */
+
+#define SPDECOMPRESS_OPCODE_CASES(CASE_MACRO) \
+ CASE_MACRO(1) \
+ CASE_MACRO(2) \
+ CASE_MACRO(4) \
+ CASE_MACRO(8) \
+ CASE_MACRO(16) \
+ CASE_MACRO(32) \
+ CASE_MACRO(64) \
+ CASE_MACRO(128)
+
+void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
+ constexpr unsigned NumFlags = 3;
+ if (N->getNumOperands() < NumFlags + 2)
+ report_fatal_error("Malformed spcompress intrinsic");
+
+ unsigned FlagOp = N->getNumOperands() - NumFlags;
+ unsigned ElemSize =
+ cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
+ unsigned IdxSize =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
+ unsigned Lg2RepeatFactor =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 2))->getZExtValue();
+
+ auto Layout = nvvm::getSPCompressLayout(ElemSize, IdxSize, Lg2RepeatFactor);
+ if (!Layout)
+ report_fatal_error(Twine("Invalid spcompress flags: elem_size=") +
+ Twine(ElemSize) + ", idx_size=" + Twine(IdxSize) +
+ ", repeat_factor=" + Twine(Lg2RepeatFactor));
+ checkSPRegTypes(N, NumFlags, "spcompress");
+
+ if (N->getNumValues() != Layout->MetadataSize + Layout->CompressedDataSize ||
+ N->getNumOperands() != Layout->DataSize + NumFlags + 2)
+ report_fatal_error("spcompress operand/result types do not match "
+ "the SP intrinsic flags");
+
+ SDLoc DL(N);
+ SmallVector<SDValue, 16> Operands;
+ Operands.reserve(N->getNumOperands() - NumFlags + 2);
+ Operands.push_back(getI32Imm(ElemSize, DL));
+ Operands.push_back(getI32Imm(IdxSize, DL));
+ Operands.push_back(getI32Imm(Lg2RepeatFactor, DL));
+ for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
+ Operands.push_back(N->getOperand(I));
+
+#define REPLACE_SPCOMPRESS_NODE(NumResults) \
+ case NumResults: \
+ ReplaceNode(N, CurDAG->getMachineNode(NVPTX::SPCOMPRESS_OUT_##NumResults, \
+ DL, N->getVTList(), Operands)); \
+ return;
+
+ switch (Layout->MetadataSize + Layout->CompressedDataSize) {
+ SPCOMPRESS_OPCODE_CASES(REPLACE_SPCOMPRESS_NODE)
+ default:
+ llvm_unreachable("invalid spcompress result count");
+ }
+
+#undef REPLACE_SPCOMPRESS_NODE
+}
+
+void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
+ unsigned NumTgt) {
+ constexpr unsigned NumFlags = 3;
+ if (N->getNumOperands() < NumFlags + 3)
+ report_fatal_error("Malformed spdecompress intrinsic");
+
+ unsigned FlagOp = N->getNumOperands() - NumFlags;
+ unsigned ElemSize =
+ cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
+ unsigned IdxSize =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
+ unsigned Lg2RepeatFactor =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 2))->getZExtValue();
+
+ auto Layout = nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSize, IdxSize,
+ Lg2RepeatFactor);
+ if (!Layout)
+ report_fatal_error(Twine("Invalid spdecompress flags: num_src=") +
+ Twine(NumSrc) + ", num_tgt=" + Twine(NumTgt) +
+ ", elem_size=" + Twine(ElemSize) +
+ ", idx_size=" + Twine(IdxSize) +
+ ", repeat_factor=" + Twine(Lg2RepeatFactor));
+ checkSPRegTypes(N, NumFlags, "spdecompress");
+
+ if (N->getNumValues() != Layout->DataSize ||
+ N->getNumOperands() !=
+ Layout->MetadataSize + Layout->CompressedDataSize + NumFlags + 1)
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
+
+ SDLoc DL(N);
+ SmallVector<SDValue, 16> Operands;
+ Operands.reserve(N->getNumOperands() - NumFlags + 4);
+ Operands.push_back(getI32Imm(ElemSize, DL));
+ Operands.push_back(getI32Imm(IdxSize, DL));
+ Operands.push_back(getI32Imm(Lg2RepeatFactor, DL));
+ Operands.push_back(getI32Imm(NumSrc, DL));
+ Operands.push_back(getI32Imm(NumTgt, DL));
+ for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
+ Operands.push_back(N->getOperand(I));
+
+#define REPLACE_SPDECOMPRESS_NODE(NumResults) \
+ case NumResults: \
+ ReplaceNode(N, \
+ CurDAG->getMachineNode(NVPTX::SPDECOMPRESS_OUT_##NumResults, \
+ DL, N->getVTList(), Operands)); \
+ return;
+
+ switch (Layout->DataSize) {
+ SPDECOMPRESS_OPCODE_CASES(REPLACE_SPDECOMPRESS_NODE)
+ default:
+ llvm_unreachable("invalid spdecompress result count");
+ }
+
+#undef REPLACE_SPDECOMPRESS_NODE
+}
+
+#undef SPDECOMPRESS_OPCODE_CASES
+#undef SPCOMPRESS_OPCODE_CASES
+
+bool NVPTXDAGToDAGISel::tryIntrinsicNoChain(SDNode *N) {
+ unsigned IID = N->getConstantOperandVal(0);
+ switch (IID) {
+ default:
+ return false;
+ case Intrinsic::nvvm_spcompress_sp2to4:
+ selectSPCompress(N);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp1to2:
+ selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/2);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp1to4:
+ selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/4);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp1to8:
+ selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/8);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp1to16:
+ selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/16);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp2to4:
+ selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/4);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp2to8:
+ selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/8);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp2to16:
+ selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/16);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp4to8:
+ selectSPDecompress(N, /*NumSrc=*/4, /*NumTgt=*/8);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp4to16:
+ selectSPDecompress(N, /*NumSrc=*/4, /*NumTgt=*/16);
+ return true;
+ }
+}
+
bool NVPTXDAGToDAGISel::tryIntrinsicChain(SDNode *N) {
unsigned IID = N->getConstantOperandVal(1);
switch (IID) {
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
index c01506f94ad90a..5ed34cb547b1b3 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
@@ -3202,6 +3202,97 @@ static SDValue lowerIntrinsicWChain(SDValue Op, SelectionDAG &DAG) {
}
}
+static bool needsSPCompressScalarization(SDNode *N) {
+ bool NeedsScalarize =
+ N->getValueType(0).isVector() || N->getValueType(1).isVector();
+ for (const SDValue &Operand : N->ops())
+ NeedsScalarize |= Operand.getValueType().isVector();
+ return NeedsScalarize;
+}
+
+static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
+ SDNode *N = Op.getNode();
+ SDLoc DL(N);
+
+ if (!needsSPCompressScalarization(N))
+ return Op;
+
+ SmallVector<SDValue, 4> Ops;
+ for (const SDValue &Operand : N->ops()) {
+ if (Operand.getValueType().isVector())
+ DAG.ExtractVectorElements(Operand, Ops);
+ else
+ Ops.push_back(Operand);
+ }
+
+ if (N->getValueType(0).isVector() || N->getValueType(1).isVector()) {
+ SmallVector<EVT, 4> ResultTys;
+ for (const EVT VT : N->values()) {
+ if (VT.isVector())
+ ResultTys.append(VT.getVectorNumElements(), VT.getVectorElementType());
+ else
+ ResultTys.push_back(VT);
+ }
+
+ SDValue Lowered = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResultTys, Ops);
+
+ SmallVector<SDValue, 2> Retvals;
+ for (unsigned NewI = 0, I = 0, E = N->getNumValues(); I != E; ++I) {
+ if (EVT VT = N->getValueType(I); VT.isVector()) {
+ SmallVector<SDValue> Elements;
+ for (auto NewE = NewI + VT.getVectorNumElements(); NewI != NewE; ++NewI)
+ Elements.push_back(Lowered.getValue(NewI));
+ Retvals.push_back(DAG.getBuildVector(VT, DL, Elements));
+ } else {
+ Retvals.push_back(Lowered.getValue(NewI));
+ ++NewI;
+ }
+ }
+
+ return DAG.getMergeValues(Retvals, DL);
+ }
+
+ return DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, N->getVTList(), Ops);
+}
+
+static SDValue lowerSPDecompress(SDValue Op, SelectionDAG &DAG) {
+ SDNode *N = Op.getNode();
+ SDLoc DL(N);
+ EVT ResVT0 = N->getValueType(0);
+ // Every valid layout with a scalar data result also has scalar mdata and
+ // cdata operands, so only vector results require scalarization here.
+ if (!ResVT0.isVector())
+ return Op;
+
+ unsigned NumElts = ResVT0.getVectorNumElements();
+ EVT EltVT = ResVT0.getVectorElementType();
+ SmallVector<EVT, 4> ListVTs(NumElts, EltVT);
+ SDVTList ResVTs = DAG.getVTList(ListVTs);
+
+ SmallVector<SDValue, 4> Ops;
+ Ops.push_back(N->getOperand(0));
+
+ for (unsigned I = 1; I < N->getNumOperands(); ++I) {
+ SDValue Op = N->getOperand(I);
+ EVT OpVT = Op.getValueType();
+ if (OpVT.isVector()) {
+ for (unsigned J = 0, E = OpVT.getVectorNumElements(); J != E; ++J)
+ Ops.push_back(DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL,
+ OpVT.getVectorElementType(), Op,
+ DAG.getIntPtrConstant(J, DL)));
+ } else {
+ Ops.push_back(Op);
+ }
+ }
+
+ SDValue NewNode = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResVTs, Ops);
+
+ SmallVector<SDValue, 4> ScalarRes;
+ for (unsigned I = 0; I < NumElts; ++I)
+ ScalarRes.push_back(NewNode.getValue(I));
+ return DAG.getBuildVector(ResVT0, DL, ScalarRes);
+}
+
static SDValue lowerIntrinsicWOChain(SDValue Op, SelectionDAG &DAG) {
switch (Op->getConstantOperandVal(0)) {
default:
@@ -3230,6 +3321,20 @@ static SDValue lowerIntrinsicWOChain(SDValue Op, SelectionDAG &DAG) {
case Intrinsic::nvvm_f32x4_to_e2m1x4_rs_satfinite:
case Intrinsic::nvvm_f32x4_to_e2m1x4_rs_relu_satfinite:
return lowerCvtRSIntrinsics(Op, DAG);
+
+ case Intrinsic::nvvm_spcompress_sp2to4:
+ return lowerSPCompress(Op, DAG);
+
+ case Intrinsic::nvvm_spdecompress_sp1to2:
+ case Intrinsic::nvvm_spdecompress_sp1to4:
+ case Intrinsic::nvvm_spdecompress_sp1to8:
+ case Intrinsic::nvvm_spdecompress_sp1to16:
+ case Intrinsic::nvvm_spdecompress_sp2to4:
+ case Intrinsic::nvvm_spdecompress_sp2to8:
+ case Intrinsic::nvvm_spdecompress_sp2to16:
+ case Intrinsic::nvvm_spdecompress_sp4to8:
+ case Intrinsic::nvvm_spdecompress_sp4to16:
+ return lowerSPDecompress(Op, DAG);
}
}
@@ -7293,6 +7398,23 @@ static SDValue combineIntrinsicWOChain(SDNode *N,
return diagnoseUnsupportedFAdd(N, DCI.DAG, IID, RoundingMode);
return combineFAddWithNeg(N, DCI.DAG, IID, RoundingMode);
}
+ case Intrinsic::nvvm_spcompress_sp2to4:
+ if (needsSPCompressScalarization(N))
+ return lowerSPCompress(SDValue(N, 0), DCI.DAG);
+ break;
+ case Intrinsic::nvvm_spdecompress_sp1to2:
+ case Intrinsic::nvvm_spdecompress_sp1to4:
+ case Intrinsic::nvvm_spdecompress_sp1to8:
+ case Intrinsic::nvvm_spdecompress_sp1to16:
+ case Intrinsic::nvvm_spdecompress_sp2to4:
+ case Intrinsic::nvvm_spdecompress_sp2to8:
+ case Intrinsic::nvvm_spdecompress_sp2to16:
+ case Intrinsic::nvvm_spdecompress_sp4to8:
+ case Intrinsic::nvvm_spdecompress_sp4to16: {
+ if (N->getValueType(0).isVector())
+ return lowerSPDecompress(SDValue(N, 0), DCI.DAG);
+ break;
+ }
}
return SDValue();
}
diff --git a/llvm/lib/Target/NVPTX/NVPTXInstrInfo.td b/llvm/lib/Target/NVPTX/NVPTXInstrInfo.td
index c130a2f351ff37..e21b5fb573a28e 100644
--- a/llvm/lib/Target/NVPTX/NVPTXInstrInfo.td
+++ b/llvm/lib/Target/NVPTX/NVPTXInstrInfo.td
@@ -185,6 +185,9 @@ def hasTensormapReplaceSupport : SubtargetPredicate;
// - conversions involving ue5m3x2
def hasRubinFamilySupport : PredOr<[SM107f]>;
+// Checks sparse compression/decompression instruction support.
+def hasSPCompress : PredOr<[SM107a]>;
+
// Checks tcgen05.shift instruction support.
def hasTcgen05ShiftSupport : PredOr<[SM100a, SM103a, SM110a]>;
diff --git a/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td b/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
index f54fd1b348af85..7e8b012c03a236 100644
--- a/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
+++ b/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
@@ -7697,6 +7697,76 @@ foreach space = ["tensor", "shared"] in {
} // cta_group
} // space
+def SPQualifier : Operand<i32> {
+ let PrintMethod = "printSPQualifier";
+}
+
+class SPRegList<int NumRegs, string Prefix> {
+ dag Dag = !dag(outs, !listsplat(B32, NumRegs),
+ !foreach(I, !range(NumRegs), Prefix # I));
+}
+
+class SPFactorName<int NumSrc, int NumTgt> {
+ string ret = ".sp::" # NumSrc # ":" # NumTgt;
+}
+
+// mdata + cdata = ceil(repeat_factor * idx_size / elem_size) + repeat_factor,
+// where the repeat_factor operand encodes the exponent.
+// The first group is shared by .b8.b2 and .b16.b4. The next groups are the
+// additional unique counts for .b16.b2 and .b8.b4.
+defvar SPCOMPRESS_OPCODE_CASES =
+ !sort(NumResultRegs,
+ !listconcat([2, 3, 5, 10, 20, 40, 80], [9, 18, 36, 72],
+ [6, 12, 24, 48, 96]),
+ NumResultRegs);
+
+// data = ceil(num_tgt * elem_size * repeat_factor / 32), where repeat_factor is
+// encoded as an exponent. The legal spdecompress layouts produce all
+// power-of-two data register counts from 1 to 128.
+defvar SPDECOMPRESS_OPCODE_CASES =
+ !listconcat(!foreach(lg2_rpf, !range(0, NVVM_SP_LG2_REPEAT_FACTOR_LIMIT),
+ !shl(1, lg2_rpf)),
+ [128]);
+
+class SPCOMPRESS_INST<int NumResultRegs, int NumSrc, int NumTgt>
+ : NVPTXInst<(outs), (ins), "?", []> {
+ let OutOperandList = SPRegList<NumResultRegs, "r">.Dag;
+ let InOperandList = (ins SPQualifier:$elem_size,
+ SPQualifier:$idx_size,
+ SPQualifier:$repeat_factor,
+ variable_ops);
+ let AsmString = "spcompress${elem_size:elem_size}${idx_size:idx_size}"
+ # SPFactorName<NumSrc, NumTgt>.ret
+ # "${repeat_factor:repeat_factor} "
+ "${elem_size:spcompress_ops};";
+ let Predicates = [hasSPCompress];
+}
+
+// SP qualifier operands control the PTX spelling and operand partitioning.
+// The opcode only encodes the number of physical result registers.
+foreach NumResultRegs = SPCOMPRESS_OPCODE_CASES in
+ def SPCOMPRESS_OUT_ # NumResultRegs : SPCOMPRESS_INST<NumResultRegs, 2, 4>;
+
+class SPDECOMPRESS_INST<int NumResultRegs>
+ : NVPTXInst<(outs), (ins), "?", []> {
+ let OutOperandList = SPRegList<NumResultRegs, "r">.Dag;
+ let InOperandList = (ins SPQualifier:$elem_size,
+ SPQualifier:$idx_size,
+ SPQualifier:$repeat_factor,
+ SPQualifier:$num_src,
+ SPQualifier:$num_tgt,
+ variable_ops);
+ let AsmString = "spdecompress${elem_size:elem_size}${idx_size:idx_size}"
+ "${num_src:spfactor}${repeat_factor:repeat_factor} "
+ "${elem_size:spdecompress_ops};";
+ let Predicates = [hasSPCompress];
+}
+
+// SP qualifier operands control the PTX spelling and operand partitioning.
+// The opcode only encodes the number of physical result registers.
+foreach NumResultRegs = SPDECOMPRESS_OPCODE_CASES in
+ def SPDECOMPRESS_OUT_ # NumResultRegs : SPDECOMPRESS_INST<NumResultRegs>;
+
//
// tensormap.replace Instructions
//
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
new file mode 100644
index 00000000000000..4f5113f0196e1b
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
@@ -0,0 +1,83 @@
+; RUN: split-file %s %t
+; RUN: not llvm-as -disable-output < %t/bad-elem-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
+; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
+; RUN: not llvm-as -disable-output < %t/bad-repeat-factor.ll 2>&1 | FileCheck %s --check-prefix=BAD-RPF
+; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
+; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
+; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
+; RUN: not --crash llc < %t/bad-result-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
+; RUN: not --crash llc < %t/bad-data-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA-TY
+
+;--- bad-elem-size.ll
+
+define { i32, i32 } @bad_spcompress_elem_size(i32 %spdesc,
+ <2 x i32> %data) {
+; BAD-ELEM: immarg value 9 for arg 2 out of range set
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 9, i32 2, i32 0)
+ ret { i32, i32 } %res
+}
+
+;--- bad-idx-size.ll
+
+define { i32, i32 } @bad_spcompress_idx_size(i32 %spdesc,
+ <2 x i32> %data) {
+; BAD-IDX: immarg value 3 for arg 3 out of range set
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 3, i32 0)
+ ret { i32, i32 } %res
+}
+
+;--- bad-repeat-factor.ll
+
+define { i32, i32 } @bad_spcompress_repeat_factor(i32 %spdesc,
+ <2 x i32> %data) {
+; BAD-RPF: immarg value 7 for arg 4 out of range [0,7)
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 7)
+ ret { i32, i32 } %res
+}
+
+;--- bad-mdata-size.ll
+
+define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
+ <8 x i32> %data) {
+; BAD-MDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %mdata = extractvalue { i32, <4 x i32> } %res, 0
+ ret i32 %mdata
+}
+
+;--- bad-cdata-size.ll
+
+define <2 x i32> @bad_spcompress_compressed_data_register_count(
+ i32 %spdesc, <8 x i32> %data) {
+; BAD-CDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { <2 x i32>, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v2i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %cdata = extractvalue { <2 x i32>, <2 x i32> } %res, 1
+ ret <2 x i32> %cdata
+}
+
+;--- bad-data-size.ll
+
+define <4 x i32> @bad_spcompress_data_register_count(i32 %spdesc,
+ <4 x i32> %data) {
+; BAD-DATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %cdata = extractvalue { <2 x i32>, <4 x i32> } %res, 1
+ ret <4 x i32> %cdata
+}
+
+;--- bad-result-type.ll
+
+define i64 @bad_spcompress_result_type(i32 %spdesc, <2 x i32> %data) {
+; BAD-RESULT-TY: LLVM ERROR: spcompress expects scalarized i32 results
+ %res = call { i64, i32 } @llvm.nvvm.spcompress.sp2to4.i64.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ %mdata = extractvalue { i64, i32 } %res, 0
+ ret i64 %mdata
+}
+
+;--- bad-data-type.ll
+
+define { i32, i32 } @bad_spcompress_data_type(i32 %spdesc, i64 %data) {
+; BAD-DATA-TY: LLVM ERROR: spcompress expects scalarized i32 operands
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.i64(i64 %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ ret { i32, i32 } %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
new file mode 100644
index 00000000000000..cc92575a3bbea6
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
@@ -0,0 +1,860 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32> %data) {
+; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<6>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x1 {%r4}, {%r5}, {%r2, %r3}, %r1;
+; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
+; CHECK-NEXT: ret;
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ ret {i32, i32} %res
+}
+
+define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32> %data) {
+; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<6>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x1 {%r4}, {%r5}, {%r2, %r3}, %r1;
+; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
+; CHECK-NEXT: ret;
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 0)
+ ret {i32, i32} %res
+}
+
+define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32> %data) {
+; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<6>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x1 {%r4}, {%r5}, {%r2, %r3}, %r1;
+; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
+; CHECK-NEXT: ret;
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 0)
+ ret {i32, i32} %res
+}
+
+define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32> %data) {
+; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<6>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x1 {%r4}, {%r5}, {%r2, %r3}, %r1;
+; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
+; CHECK-NEXT: ret;
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 0)
+ ret {i32, i32} %res
+}
+
+define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <4 x i32> %data) {
+; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<9>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x2 {%r6}, {%r7, %r8}, {%r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
+; CHECK-NEXT: ret;
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 1)
+ ret {i32, <2 x i32>} %res
+}
+
+define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <4 x i32> %data) {
+; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<9>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x2 {%r6}, {%r7, %r8}, {%r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
+; CHECK-NEXT: ret;
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 1)
+ ret {i32, <2 x i32>} %res
+}
+
+define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <4 x i32> %data) {
+; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<9>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x2 {%r6}, {%r7, %r8}, {%r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
+; CHECK-NEXT: ret;
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 1)
+ ret {i32, <2 x i32>} %res
+}
+
+define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <4 x i32> %data) {
+; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<9>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x2 {%r6}, {%r7, %r8}, {%r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
+; CHECK-NEXT: ret;
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 1)
+ ret {i32, <2 x i32>} %res
+}
+
+define {i32, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <8 x i32> %data) {
+; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_4_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x4 {%r10}, {%r11, %r12, %r13, %r14}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
+; CHECK-NEXT: ret;
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 2)
+ ret {i32, <4 x i32>} %res
+}
+
+define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <8 x i32> %data) {
+; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_4_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x4 {%r10}, {%r11, %r12, %r13, %r14}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
+; CHECK-NEXT: ret;
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 2)
+ ret {i32, <4 x i32>} %res
+}
+
+define {<2 x i32>, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spdesc, <8 x i32> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <4 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<16>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_4_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x4 {%r10, %r11}, {%r12, %r13, %r14, %r15}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r10, %r11};
+; CHECK-NEXT: ret;
+ %res = call {<2 x i32>, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ ret {<2 x i32>, <4 x i32>} %res
+}
+
+define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <8 x i32> %data) {
+; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_4_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x4 {%r10}, {%r11, %r12, %r13, %r14}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
+; CHECK-NEXT: ret;
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 2)
+ ret {i32, <4 x i32>} %res
+}
+
+define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spdesc, <16 x i32> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <8 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<28>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x8 {%r18, %r19}, {%r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27}, {%r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r24, %r25, %r26, %r27};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
+; CHECK-NEXT: ret;
+ %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 3)
+ ret {<2 x i32>, <8 x i32>} %res
+}
+
+define {i32, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <16 x i32> %data) {
+; FORMAT-LABEL: define { i32, <8 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call { i32, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x8 {%r18}, {%r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r18;
+; CHECK-NEXT: ret;
+ %res = call {i32, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 3)
+ ret {i32, <8 x i32>} %res
+}
+
+define {<4 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spdesc, <16 x i32> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <8 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call { <4 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<30>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x8 {%r18, %r19, %r20, %r21}, {%r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29}, {%r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: ret;
+ %res = call {<4 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 3)
+ ret {<4 x i32>, <8 x i32>} %res
+}
+
+define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spdesc, <16 x i32> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <8 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<28>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x8 {%r18, %r19}, {%r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27}, {%r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r24, %r25, %r26, %r27};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
+; CHECK-NEXT: ret;
+ %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 3)
+ ret {<2 x i32>, <8 x i32>} %res
+}
+
+define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spdesc, <32 x i32> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <16 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<54>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x16 {%r34, %r35, %r36, %r37}, {%r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53}, {%r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r50, %r51, %r52, %r53};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r46, %r47, %r48, %r49};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r42, %r43, %r44, %r45};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: ret;
+ %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 4)
+ ret {<4 x i32>, <16 x i32>} %res
+}
+
+define {<2 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %spdesc, <32 x i32> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <16 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<52>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x16 {%r34, %r35}, {%r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51}, {%r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r48, %r49, %r50, %r51};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r44, %r45, %r46, %r47};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r40, %r41, %r42, %r43};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r36, %r37, %r38, %r39};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r34, %r35};
+; CHECK-NEXT: ret;
+ %res = call {<2 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 4)
+ ret {<2 x i32>, <16 x i32>} %res
+}
+
+define {<8 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spdesc, <32 x i32> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <16 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call { <8 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<58>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x16 {%r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41}, {%r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57}, {%r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r54, %r55, %r56, %r57};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r50, %r51, %r52, %r53};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r46, %r47, %r48, %r49};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r42, %r43, %r44, %r45};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: ret;
+ %res = call {<8 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 4)
+ ret {<8 x i32>, <16 x i32>} %res
+}
+
+define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %spdesc, <32 x i32> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <16 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<54>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x16 {%r34, %r35, %r36, %r37}, {%r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53}, {%r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r50, %r51, %r52, %r53};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r46, %r47, %r48, %r49};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r42, %r43, %r44, %r45};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: ret;
+ %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 4)
+ ret {<4 x i32>, <16 x i32>} %res
+}
+
+define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %spdesc, <64 x i32> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <32 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<106>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x32 {%r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73}, {%r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105}, {%r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r102, %r103, %r104, %r105};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r98, %r99, %r100, %r101};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r94, %r95, %r96, %r97};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r90, %r91, %r92, %r93};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r86, %r87, %r88, %r89};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r82, %r83, %r84, %r85};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r78, %r79, %r80, %r81};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r74, %r75, %r76, %r77};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: ret;
+ %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 5)
+ ret {<8 x i32>, <32 x i32>} %res
+}
+
+define {<4 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %spdesc, <64 x i32> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <32 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<102>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x32 {%r66, %r67, %r68, %r69}, {%r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101}, {%r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r98, %r99, %r100, %r101};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r94, %r95, %r96, %r97};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r90, %r91, %r92, %r93};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r86, %r87, %r88, %r89};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r82, %r83, %r84, %r85};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r78, %r79, %r80, %r81};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r74, %r75, %r76, %r77};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r70, %r71, %r72, %r73};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: ret;
+ %res = call {<4 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 5)
+ ret {<4 x i32>, <32 x i32>} %res
+}
+
+define {<16 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %spdesc, <64 x i32> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <32 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call { <16 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<114>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x32 {%r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81}, {%r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113}, {%r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r110, %r111, %r112, %r113};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r106, %r107, %r108, %r109};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r102, %r103, %r104, %r105};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r98, %r99, %r100, %r101};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r94, %r95, %r96, %r97};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r90, %r91, %r92, %r93};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r86, %r87, %r88, %r89};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r82, %r83, %r84, %r85};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r78, %r79, %r80, %r81};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r74, %r75, %r76, %r77};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: ret;
+ %res = call {<16 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 5)
+ ret {<16 x i32>, <32 x i32>} %res
+}
+
+define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %spdesc, <64 x i32> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <32 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<106>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x32 {%r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73}, {%r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105}, {%r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r102, %r103, %r104, %r105};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r98, %r99, %r100, %r101};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r94, %r95, %r96, %r97};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r90, %r91, %r92, %r93};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r86, %r87, %r88, %r89};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r82, %r83, %r84, %r85};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r78, %r79, %r80, %r81};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r74, %r75, %r76, %r77};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: ret;
+ %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 5)
+ ret {<8 x i32>, <32 x i32>} %res
+}
+
+define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %spdesc, <128 x i32> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <64 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<210>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+496];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+480];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+464];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+448];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+432];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+416];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+400];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+384];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+368];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+352];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+336];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+320];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+304];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+288];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+272];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+256];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r66, %r67, %r68, %r69}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r70, %r71, %r72, %r73}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r74, %r75, %r76, %r77}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r78, %r79, %r80, %r81}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r82, %r83, %r84, %r85}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r86, %r87, %r88, %r89}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r90, %r91, %r92, %r93}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r94, %r95, %r96, %r97}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r98, %r99, %r100, %r101}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r102, %r103, %r104, %r105}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r106, %r107, %r108, %r109}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r110, %r111, %r112, %r113}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r114, %r115, %r116, %r117}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r118, %r119, %r120, %r121}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r122, %r123, %r124, %r125}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r126, %r127, %r128, %r129}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x64 {%r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145}, {%r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208, %r209}, {%r126, %r127, %r128, %r129, %r122, %r123, %r124, %r125, %r118, %r119, %r120, %r121, %r114, %r115, %r116, %r117, %r110, %r111, %r112, %r113, %r106, %r107, %r108, %r109, %r102, %r103, %r104, %r105, %r98, %r99, %r100, %r101, %r94, %r95, %r96, %r97, %r90, %r91, %r92, %r93, %r86, %r87, %r88, %r89, %r82, %r83, %r84, %r85, %r78, %r79, %r80, %r81, %r74, %r75, %r76, %r77, %r70, %r71, %r72, %r73, %r66, %r67, %r68, %r69, %r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r206, %r207, %r208, %r209};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r202, %r203, %r204, %r205};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r198, %r199, %r200, %r201};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r194, %r195, %r196, %r197};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r190, %r191, %r192, %r193};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r186, %r187, %r188, %r189};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r182, %r183, %r184, %r185};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r178, %r179, %r180, %r181};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r174, %r175, %r176, %r177};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r170, %r171, %r172, %r173};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r166, %r167, %r168, %r169};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r162, %r163, %r164, %r165};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r158, %r159, %r160, %r161};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r154, %r155, %r156, %r157};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r150, %r151, %r152, %r153};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r146, %r147, %r148, %r149};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r142, %r143, %r144, %r145};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r138, %r139, %r140, %r141};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
+; CHECK-NEXT: ret;
+ %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 6)
+ ret {<16 x i32>, <64 x i32>} %res
+}
+
+define {<8 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %spdesc, <128 x i32> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <64 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<202>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+496];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+480];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+464];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+448];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+432];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+416];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+400];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+384];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+368];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+352];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+336];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+320];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+304];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+288];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+272];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+256];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r66, %r67, %r68, %r69}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r70, %r71, %r72, %r73}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r74, %r75, %r76, %r77}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r78, %r79, %r80, %r81}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r82, %r83, %r84, %r85}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r86, %r87, %r88, %r89}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r90, %r91, %r92, %r93}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r94, %r95, %r96, %r97}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r98, %r99, %r100, %r101}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r102, %r103, %r104, %r105}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r106, %r107, %r108, %r109}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r110, %r111, %r112, %r113}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r114, %r115, %r116, %r117}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r118, %r119, %r120, %r121}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r122, %r123, %r124, %r125}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r126, %r127, %r128, %r129}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x64 {%r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137}, {%r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201}, {%r126, %r127, %r128, %r129, %r122, %r123, %r124, %r125, %r118, %r119, %r120, %r121, %r114, %r115, %r116, %r117, %r110, %r111, %r112, %r113, %r106, %r107, %r108, %r109, %r102, %r103, %r104, %r105, %r98, %r99, %r100, %r101, %r94, %r95, %r96, %r97, %r90, %r91, %r92, %r93, %r86, %r87, %r88, %r89, %r82, %r83, %r84, %r85, %r78, %r79, %r80, %r81, %r74, %r75, %r76, %r77, %r70, %r71, %r72, %r73, %r66, %r67, %r68, %r69, %r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r198, %r199, %r200, %r201};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r194, %r195, %r196, %r197};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r190, %r191, %r192, %r193};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r186, %r187, %r188, %r189};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r182, %r183, %r184, %r185};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r178, %r179, %r180, %r181};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r174, %r175, %r176, %r177};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r170, %r171, %r172, %r173};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r166, %r167, %r168, %r169};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r162, %r163, %r164, %r165};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r158, %r159, %r160, %r161};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r154, %r155, %r156, %r157};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r150, %r151, %r152, %r153};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r146, %r147, %r148, %r149};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r142, %r143, %r144, %r145};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r138, %r139, %r140, %r141};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
+; CHECK-NEXT: ret;
+ %res = call {<8 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 6)
+ ret {<8 x i32>, <64 x i32>} %res
+}
+
+define {<32 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %spdesc, <128 x i32> %data) {
+; FORMAT-LABEL: define { <32 x i32>, <64 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call { <32 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<226>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+496];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+480];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+464];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+448];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+432];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+416];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+400];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+384];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+368];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+352];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+336];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+320];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+304];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+288];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+272];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+256];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r66, %r67, %r68, %r69}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r70, %r71, %r72, %r73}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r74, %r75, %r76, %r77}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r78, %r79, %r80, %r81}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r82, %r83, %r84, %r85}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r86, %r87, %r88, %r89}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r90, %r91, %r92, %r93}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r94, %r95, %r96, %r97}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r98, %r99, %r100, %r101}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r102, %r103, %r104, %r105}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r106, %r107, %r108, %r109}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r110, %r111, %r112, %r113}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r114, %r115, %r116, %r117}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r118, %r119, %r120, %r121}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r122, %r123, %r124, %r125}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r126, %r127, %r128, %r129}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x64 {%r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161}, {%r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208, %r209, %r210, %r211, %r212, %r213, %r214, %r215, %r216, %r217, %r218, %r219, %r220, %r221, %r222, %r223, %r224, %r225}, {%r126, %r127, %r128, %r129, %r122, %r123, %r124, %r125, %r118, %r119, %r120, %r121, %r114, %r115, %r116, %r117, %r110, %r111, %r112, %r113, %r106, %r107, %r108, %r109, %r102, %r103, %r104, %r105, %r98, %r99, %r100, %r101, %r94, %r95, %r96, %r97, %r90, %r91, %r92, %r93, %r86, %r87, %r88, %r89, %r82, %r83, %r84, %r85, %r78, %r79, %r80, %r81, %r74, %r75, %r76, %r77, %r70, %r71, %r72, %r73, %r66, %r67, %r68, %r69, %r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r222, %r223, %r224, %r225};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r218, %r219, %r220, %r221};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r214, %r215, %r216, %r217};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r210, %r211, %r212, %r213};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r206, %r207, %r208, %r209};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r202, %r203, %r204, %r205};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r198, %r199, %r200, %r201};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r194, %r195, %r196, %r197};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r190, %r191, %r192, %r193};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r186, %r187, %r188, %r189};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r182, %r183, %r184, %r185};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r178, %r179, %r180, %r181};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r174, %r175, %r176, %r177};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r170, %r171, %r172, %r173};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r166, %r167, %r168, %r169};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r162, %r163, %r164, %r165};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r158, %r159, %r160, %r161};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r154, %r155, %r156, %r157};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r150, %r151, %r152, %r153};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r146, %r147, %r148, %r149};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r142, %r143, %r144, %r145};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r138, %r139, %r140, %r141};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
+; CHECK-NEXT: ret;
+ %res = call {<32 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 6)
+ ret {<32 x i32>, <64 x i32>} %res
+}
+
+define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %spdesc, <128 x i32> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <64 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<210>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+496];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+480];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+464];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+448];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+432];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+416];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+400];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+384];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+368];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+352];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+336];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+320];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+304];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+288];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+272];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+256];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r66, %r67, %r68, %r69}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r70, %r71, %r72, %r73}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r74, %r75, %r76, %r77}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r78, %r79, %r80, %r81}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r82, %r83, %r84, %r85}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r86, %r87, %r88, %r89}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r90, %r91, %r92, %r93}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r94, %r95, %r96, %r97}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r98, %r99, %r100, %r101}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r102, %r103, %r104, %r105}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r106, %r107, %r108, %r109}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r110, %r111, %r112, %r113}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r114, %r115, %r116, %r117}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r118, %r119, %r120, %r121}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r122, %r123, %r124, %r125}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r126, %r127, %r128, %r129}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x64 {%r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145}, {%r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208, %r209}, {%r126, %r127, %r128, %r129, %r122, %r123, %r124, %r125, %r118, %r119, %r120, %r121, %r114, %r115, %r116, %r117, %r110, %r111, %r112, %r113, %r106, %r107, %r108, %r109, %r102, %r103, %r104, %r105, %r98, %r99, %r100, %r101, %r94, %r95, %r96, %r97, %r90, %r91, %r92, %r93, %r86, %r87, %r88, %r89, %r82, %r83, %r84, %r85, %r78, %r79, %r80, %r81, %r74, %r75, %r76, %r77, %r70, %r71, %r72, %r73, %r66, %r67, %r68, %r69, %r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r206, %r207, %r208, %r209};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r202, %r203, %r204, %r205};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r198, %r199, %r200, %r201};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r194, %r195, %r196, %r197};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r190, %r191, %r192, %r193};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r186, %r187, %r188, %r189};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r182, %r183, %r184, %r185};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r178, %r179, %r180, %r181};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r174, %r175, %r176, %r177};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r170, %r171, %r172, %r173};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r166, %r167, %r168, %r169};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r162, %r163, %r164, %r165};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r158, %r159, %r160, %r161};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r154, %r155, %r156, %r157};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r150, %r151, %r152, %r153};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r146, %r147, %r148, %r149};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r142, %r143, %r144, %r145};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r138, %r139, %r140, %r141};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
+; CHECK-NEXT: ret;
+ %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 6)
+ ret {<16 x i32>, <64 x i32>} %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
new file mode 100644
index 00000000000000..74d6eaa53dc66d
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
@@ -0,0 +1,105 @@
+; RUN: split-file %s %t
+; RUN: not llvm-as -disable-output < %t/bad-elem-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
+; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
+; RUN: not llvm-as -disable-output < %t/bad-repeat-factor.ll 2>&1 | FileCheck %s --check-prefix=BAD-RPF
+; RUN: not --crash llc < %t/bad-idx-size-for-target.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-IDX-TARGET
+; RUN: not --crash llc < %t/bad-input-span.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-INPUT-SPAN
+; RUN: not --crash llc < %t/bad-data-too-small.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA-SMALL
+; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
+; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
+; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
+; RUN: not --crash llc < %t/bad-result-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
+; RUN: not --crash llc < %t/bad-mdata-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA-TY
+
+;--- bad-elem-size.ll
+
+define i32 @bad_spdecompress_elem_size(i32 %metadata, i32 %compdata) {
+; BAD-ELEM: immarg value 9 for arg 2 out of range set
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 9, i32 2, i32 0)
+ ret i32 %res
+}
+
+;--- bad-idx-size.ll
+
+define i32 @bad_spdecompress_idx_size(i32 %metadata, i32 %compdata) {
+; BAD-IDX: immarg value 3 for arg 3 out of range set
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 3, i32 0)
+ ret i32 %res
+}
+
+;--- bad-repeat-factor.ll
+
+define i32 @bad_spdecompress_repeat_factor(i32 %metadata, i32 %compdata) {
+; BAD-RPF: immarg value 7 for arg 4 out of range [0,7)
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 7)
+ ret i32 %res
+}
+
+;--- bad-idx-size-for-target.ll
+
+define <8 x i32> @bad_spdecompress_idx_size_for_target(
+ i32 %metadata, i32 %compdata) {
+; BAD-IDX-TARGET: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=8, elem_size=8, idx_size=2, repeat_factor=2
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+ ret <8 x i32> %res
+}
+
+;--- bad-input-span.ll
+
+define <4 x i32> @bad_spdecompress_input_span(<2 x i32> %metadata,
+ <2 x i32> %compdata) {
+; BAD-INPUT-SPAN: LLVM ERROR: Invalid spdecompress flags: num_src=4, num_tgt=8, elem_size=16, idx_size=4, repeat_factor=0
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+;--- bad-data-too-small.ll
+
+define i32 @bad_spdecompress_data_too_small(i32 %metadata, i32 %compdata) {
+; BAD-DATA-SMALL: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=2, elem_size=8, idx_size=2, repeat_factor=0
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i32 %res
+}
+
+;--- bad-mdata-size.ll
+
+define <4 x i32> @bad_spdecompress_metadata_register_count(
+ <2 x i32> %metadata, <2 x i32> %compdata) {
+; BAD-MDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <4 x i32> %res
+}
+
+;--- bad-cdata-size.ll
+
+define <4 x i32> @bad_spdecompress_compressed_data_register_count(
+ i32 %metadata, i32 %compdata) {
+; BAD-CDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+;--- bad-data-size.ll
+
+define <2 x i32> @bad_spdecompress_data_register_count(
+ i32 %metadata, <2 x i32> %compdata) {
+; BAD-DATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <2 x i32> %res
+}
+
+;--- bad-result-type.ll
+
+define i64 @bad_spdecompress_result_type(i32 %metadata, i32 %compdata) {
+; BAD-RESULT-TY: LLVM ERROR: spdecompress expects scalarized i32 results
+ %res = call i64 @llvm.nvvm.spdecompress.sp2to4.i64.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i64 %res
+}
+
+;--- bad-mdata-type.ll
+
+define i32 @bad_spdecompress_metadata_type(i64 %metadata, i32 %compdata) {
+; BAD-MDATA-TY: LLVM ERROR: spdecompress expects scalarized i32 operands
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i64.i32(i64 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i32 %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
new file mode 100644
index 00000000000000..e375269b36e494
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
@@ -0,0 +1,309 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4
+define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<19>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x4 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<36>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x8 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r32, %r33, %r34, %r35};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r28, %r29, %r30, %r31};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r24, %r25, %r26, %r27};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r20, %r21, %r22, %r23};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<71>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x16 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<141>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x32 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4
+define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<19>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<36>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r32, %r33, %r34, %r35};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r28, %r29, %r30, %r31};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r24, %r25, %r26, %r27};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r20, %r21, %r22, %r23};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<70>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r62, %r63, %r64, %r65};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r58, %r59, %r60, %r61};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r54, %r55, %r56, %r57};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r50, %r51, %r52, %r53};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r46, %r47, %r48, %r49};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r42, %r43, %r44, %r45};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r30, %r31, %r32, %r33};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<139>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r135, %r136, %r137, %r138};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r131, %r132, %r133, %r134};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r127, %r128, %r129, %r130};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r123, %r124, %r125, %r126};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r119, %r120, %r121, %r122};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r115, %r116, %r117, %r118};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r111, %r112, %r113, %r114};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r107, %r108, %r109, %r110};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r103, %r104, %r105, %r106};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r99, %r100, %r101, %r102};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r95, %r96, %r97, %r98};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r91, %r92, %r93, %r94};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r87, %r88, %r89, %r90};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r83, %r84, %r85, %r86};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r79, %r80, %r81, %r82};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r75, %r76, %r77, %r78};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r71, %r72, %r73, %r74};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
new file mode 100644
index 00000000000000..e90aed6e35fb07
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
@@ -0,0 +1,552 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x2 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 3)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x16 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 4)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x32 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 5)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<53>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x64 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 6)
+ ret <32 x i32> %res
+}
+
+define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x2 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x16 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<29>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x32 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<57>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x64 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <32 x i32> %res
+}
+
+define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 1)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 2)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 3)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<26>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x16 {%r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25}, {%r1}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 4)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<51>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r11, %r12, %r13, %r14}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r15, %r16, %r17, %r18}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x32 {%r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50}, {%r1, %r2}, {%r15, %r16, %r17, %r18, %r11, %r12, %r13, %r14, %r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 5)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<101>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x64 {%r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100}, {%r1, %r2, %r3, %r4}, {%r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 6)
+ ret <64 x i32> %res
+}
+
+define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<53>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x32 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<105>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x64 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 6)
+ ret <64 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
new file mode 100644
index 00000000000000..c9c369e724a074
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
@@ -0,0 +1,682 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=2
+define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x8 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 3)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<22>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x16 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 4)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<43>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x32 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 5)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<85>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x64 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 6)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4
+define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x8 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<23>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x16 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<45>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x32 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<89>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x64 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=2
+define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 1)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 2)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<22>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 3)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<42>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x16 {%r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41}, {%r1}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r30, %r31, %r32, %r33};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 4)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<83>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r11, %r12, %r13, %r14}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r15, %r16, %r17, %r18}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x32 {%r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82}, {%r1, %r2}, {%r15, %r16, %r17, %r18, %r11, %r12, %r13, %r14, %r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r79, %r80, %r81, %r82};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r75, %r76, %r77, %r78};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r71, %r72, %r73, %r74};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 5)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<165>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x64 {%r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164}, {%r1, %r2, %r3, %r4}, {%r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 6)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4
+define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<22>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<43>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<85>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x32 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<169>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x64 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 6)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
new file mode 100644
index 00000000000000..c9c5347fa485e6
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
@@ -0,0 +1,350 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4
+define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x4 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<20>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x8 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<39>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x16 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<77>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x32 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<153>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x64 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4
+define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<20>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<38>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r30, %r31, %r32, %r33};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<75>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r71, %r72, %r73, %r74};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<149>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x32 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
new file mode 100644
index 00000000000000..a85bda8ebb5e47
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
@@ -0,0 +1,325 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4, rpf=1
+define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<20>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<39>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x8 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<77>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x16 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<153>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x32 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=1
+define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x1 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <8 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=2
+define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<20>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x2 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ ret <16 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=4
+define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<38>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x4 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r30, %r31, %r32, %r33};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <32 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=8
+define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<75>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x8 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r71, %r72, %r73, %r74};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=16
+define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<149>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x16 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
new file mode 100644
index 00000000000000..d332a2e5fc790e
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
@@ -0,0 +1,758 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=2, rpf=1
+define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i32 %res
+}
+
+; elem_size=8, idx_size=2, rpf=2
+define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ ret <2 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=4
+define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 2)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=8
+define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 3)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=16
+define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 4)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=32
+define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<53>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x32 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 5)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=64
+define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<105>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x64 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 2, i32 6)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=1
+define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret i32 %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <2 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x8 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<29>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x16 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<57>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x32 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=64
+define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<113>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x64 {%r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=1
+define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ ret <2 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=2
+define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 1)
+ ret <4 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=4
+define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x4 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 2)
+ ret <8 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=8
+define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<26>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x8 {%r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25}, {%r1}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 3)
+ ret <16 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=16
+define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<51>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r11, %r12, %r13, %r14}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r15, %r16, %r17, %r18}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x16 {%r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50}, {%r1, %r2}, {%r15, %r16, %r17, %r18, %r11, %r12, %r13, %r14, %r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 4)
+ ret <32 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=32
+define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<101>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x32 {%r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100}, {%r1, %r2, %r3, %r4}, {%r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 5)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=64
+define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32> %metadata, <64 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<201>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r49, %r50, %r51, %r52}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r53, %r54, %r55, %r56}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r57, %r58, %r59, %r60}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r61, %r62, %r63, %r64}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r65, %r66, %r67, %r68}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r69, %r70, %r71, %r72}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x64 {%r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r69, %r70, %r71, %r72, %r65, %r66, %r67, %r68, %r61, %r62, %r63, %r64, %r57, %r58, %r59, %r60, %r53, %r54, %r55, %r56, %r49, %r50, %r51, %r52, %r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r197, %r198, %r199, %r200};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r193, %r194, %r195, %r196};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r189, %r190, %r191, %r192};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r185, %r186, %r187, %r188};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r181, %r182, %r183, %r184};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r177, %r178, %r179, %r180};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 2, i32 6)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=1
+define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=2
+define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=4
+define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x4 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=8
+define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x8 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=16
+define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<53>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x16 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=32
+define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<105>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x32 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=64
+define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32> %metadata, <64 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<209>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r49, %r50, %r51, %r52}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r53, %r54, %r55, %r56}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r57, %r58, %r59, %r60}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r61, %r62, %r63, %r64}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r65, %r66, %r67, %r68}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r69, %r70, %r71, %r72}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r73, %r74, %r75, %r76}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r77, %r78, %r79, %r80}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x64 {%r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r77, %r78, %r79, %r80, %r73, %r74, %r75, %r76, %r69, %r70, %r71, %r72, %r65, %r66, %r67, %r68, %r61, %r62, %r63, %r64, %r57, %r58, %r59, %r60, %r53, %r54, %r55, %r56, %r49, %r50, %r51, %r52, %r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r205, %r206, %r207, %r208};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r201, %r202, %r203, %r204};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r197, %r198, %r199, %r200};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r193, %r194, %r195, %r196};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r189, %r190, %r191, %r192};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r185, %r186, %r187, %r188};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r181, %r182, %r183, %r184};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r177, %r178, %r179, %r180};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 4, i32 6)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
new file mode 100644
index 00000000000000..452efc202134fd
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
@@ -0,0 +1,379 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4, rpf=1
+define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<23>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x8 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<45>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x16 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<89>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x32 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=64
+define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<177>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x64 {%r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=1
+define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=2
+define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x2 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=4
+define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<22>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x4 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=8
+define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<43>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x8 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=16
+define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<85>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x16 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=32
+define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<169>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x32 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
new file mode 100644
index 00000000000000..ad2f926a520859
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
@@ -0,0 +1,184 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4, rpf=1
+define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x2 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<23>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x4 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<45>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x8 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<89>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x16 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<177>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x32 {%r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
new file mode 100644
index 00000000000000..94bff5c02f6f2a
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
@@ -0,0 +1,224 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4, rpf=1
+define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x4 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<29>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x8 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<57>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x16 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<113>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x32 {%r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=64
+define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32> %metadata, <64 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<225>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r49, %r50, %r51, %r52}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r53, %r54, %r55, %r56}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r57, %r58, %r59, %r60}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r61, %r62, %r63, %r64}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r65, %r66, %r67, %r68}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r69, %r70, %r71, %r72}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r73, %r74, %r75, %r76}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r77, %r78, %r79, %r80}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r81, %r82, %r83, %r84}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r85, %r86, %r87, %r88}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r89, %r90, %r91, %r92}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r93, %r94, %r95, %r96}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x64 {%r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208, %r209, %r210, %r211, %r212, %r213, %r214, %r215, %r216, %r217, %r218, %r219, %r220, %r221, %r222, %r223, %r224}, {%r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r93, %r94, %r95, %r96, %r89, %r90, %r91, %r92, %r85, %r86, %r87, %r88, %r81, %r82, %r83, %r84, %r77, %r78, %r79, %r80, %r73, %r74, %r75, %r76, %r69, %r70, %r71, %r72, %r65, %r66, %r67, %r68, %r61, %r62, %r63, %r64, %r57, %r58, %r59, %r60, %r53, %r54, %r55, %r56, %r49, %r50, %r51, %r52, %r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r221, %r222, %r223, %r224};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r217, %r218, %r219, %r220};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r213, %r214, %r215, %r216};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r209, %r210, %r211, %r212};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r205, %r206, %r207, %r208};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r201, %r202, %r203, %r204};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r197, %r198, %r199, %r200};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r193, %r194, %r195, %r196};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r189, %r190, %r191, %r192};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r185, %r186, %r187, %r188};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r181, %r182, %r183, %r184};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r177, %r178, %r179, %r180};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <128 x i32> %res
+}
>From ae06763a90ee4cfb8aba3baa8a5a9eae68398df8 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Wed, 9 Sep 2026 16:48:04 +0200
Subject: [PATCH 02/10] [NVVM][NVPTX] Infer sparse intrinsic repeat factors
from types
---
llvm/docs/NVPTXUsage.md | 17 +--
llvm/include/llvm/IR/IntrinsicsNVVM.td | 20 +---
llvm/include/llvm/IR/NVVMIntrinsicUtils.h | 49 ++++----
llvm/lib/IR/NVVMIntrinsicUtils.cpp | 6 -
.../NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp | 6 +-
llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp | 46 ++++---
llvm/lib/Target/NVPTX/NVPTXIntrinsics.td | 14 +--
llvm/test/CodeGen/NVPTX/spcompress-invalid.ll | 26 ++--
llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll | 112 +++++++++---------
.../CodeGen/NVPTX/spdecompress-invalid.ll | 44 ++-----
.../CodeGen/NVPTX/spdecompress-sp1to16.ll | 44 +++----
.../test/CodeGen/NVPTX/spdecompress-sp1to2.ll | 104 ++++++++--------
.../test/CodeGen/NVPTX/spdecompress-sp1to4.ll | 112 +++++++++---------
.../test/CodeGen/NVPTX/spdecompress-sp1to8.ll | 52 ++++----
.../CodeGen/NVPTX/spdecompress-sp2to16.ll | 44 +++----
.../test/CodeGen/NVPTX/spdecompress-sp2to4.ll | 112 +++++++++---------
.../test/CodeGen/NVPTX/spdecompress-sp2to8.ll | 52 ++++----
.../CodeGen/NVPTX/spdecompress-sp4to16.ll | 24 ++--
.../test/CodeGen/NVPTX/spdecompress-sp4to8.ll | 28 ++---
19 files changed, 438 insertions(+), 474 deletions(-)
diff --git a/llvm/docs/NVPTXUsage.md b/llvm/docs/NVPTXUsage.md
index a048ed047c46cc..31b140b722e69a 100644
--- a/llvm/docs/NVPTXUsage.md
+++ b/llvm/docs/NVPTXUsage.md
@@ -1039,16 +1039,15 @@ A bundle of one 32-bit register is represented by `i32`, and a bundle of `N`
registers is represented by `<N x i32>`. The actual intrinsic names include
the corresponding LLVM overload suffixes.
-The three trailing arguments are immediate qualifiers with the following
+The two trailing arguments are immediate qualifiers with the following
encodings:
| Argument | Values | PTX qualifiers |
| ----------------------- | ------------------------- | -------------------------------------- |
| `%elem_size` | `8`, `16` | `.b8`, `.b16` |
| `%idx_size` | `2`, `4` | `.b2`, `.b4` |
-| `%repeat_factor` | `0`, `1`, ..., `6` | `.x1`, `.x2`, ..., `.x64` |
-The repeat factor `num` is `1 << %repeat_factor`.
+The PTX repeat-factor qualifier is inferred from the overloaded bundle types.
#### '`llvm.nvvm.spcompress.sp2to4`' Intrinsic
@@ -1056,8 +1055,7 @@ The repeat factor `num` is `1 << %repeat_factor`.
```llvm
declare {MDataTy, CDataTy} @llvm.nvvm.spcompress.sp2to4(
- DataTy %data, i32 %spdesc, i32 immarg %elem_size,
- i32 immarg %idx_size, i32 immarg %repeat_factor)
+ DataTy %data, i32 %spdesc, i32 immarg %elem_size, i32 immarg %idx_size)
```
##### Overview:
@@ -1072,6 +1070,9 @@ elements as `cdata`. The number of 32-bit registers in each bundle is:
| `cdata` | `num` |
| `mdata` | `ceil(num * %idx_size / %elem_size)` |
+Here, `num` is half the number of registers in `DataTy` and determines the PTX
+repeat-factor qualifier (`.x1`, `.x2`, ..., `.x64`).
+
The combined `mdata`, `cdata`, and `data` bundle size must not exceed 253
registers.
@@ -1101,8 +1102,7 @@ For more information, see the
declare DataTy @llvm.nvvm.spdecompress.{sp1to2,sp1to4,sp1to8,sp1to16,
sp2to4,sp2to8,sp2to16,
sp4to8,sp4to16}(
- MDataTy %mdata, CDataTy %cdata, i32 immarg %elem_size,
- i32 immarg %idx_size, i32 immarg %repeat_factor)
+ MDataTy %mdata, CDataTy %cdata, i32 immarg %elem_size, i32 immarg %idx_size)
```
##### Overview:
@@ -1122,6 +1122,9 @@ is:
| `cdata` | `ceil(X * %elem_size * num / 32)` |
| `data` | `ceil(Y * %elem_size * num / 32)` |
+Here, `num` is inferred from the number of registers in `DataTy` and determines
+the PTX repeat-factor qualifier (`.x1`, `.x2`, ..., `.x64`).
+
The following conditions must hold:
- `X * %elem_size <= 32`.
diff --git a/llvm/include/llvm/IR/IntrinsicsNVVM.td b/llvm/include/llvm/IR/IntrinsicsNVVM.td
index d3dc735785552f..66948a121ffb40 100644
--- a/llvm/include/llvm/IR/IntrinsicsNVVM.td
+++ b/llvm/include/llvm/IR/IntrinsicsNVVM.td
@@ -1335,20 +1335,14 @@ class NVVMPureIntrinsic<list<LLVMType> ret_types,
DefaultAttrsIntrinsic<ret_types, param_types,
intr_properties # [IntrNoMem, IntrSpeculatable], name> {}
-defvar NVVM_SP_LG2_REPEAT_FACTOR_LIMIT = 7;
-
-class NVVM_SP_ARGPROPS<ArgIndex ElemSizeIdx, ArgIndex IdxSizeIdx,
- ArgIndex RepeatFactorIdx> {
+class NVVM_SP_ARGPROPS<ArgIndex ElemSizeIdx, ArgIndex IdxSizeIdx> {
list<IntrinsicProperty> Prop = [
RangeList<ElemSizeIdx, [8, 16]>,
RangeList<IdxSizeIdx, [2, 4]>,
- Range<RepeatFactorIdx, 0, NVVM_SP_LG2_REPEAT_FACTOR_LIMIT>,
ArgInfo<ElemSizeIdx, [ArgName<"elem_size">,
ImmArgPrinter<"printSPElemSize">]>,
ArgInfo<IdxSizeIdx, [ArgName<"idx_size">,
- ImmArgPrinter<"printSPIdxSize">]>,
- ArgInfo<RepeatFactorIdx, [ArgName<"repeat_factor">,
- ImmArgPrinter<"printSPRepeatFactor">]>
+ ImmArgPrinter<"printSPIdxSize">]>
];
}
@@ -4246,10 +4240,9 @@ class SPCompress :
[llvm_anyint_ty, // data
llvm_i32_ty], // spdesc
[llvm_i32_ty, // elem_size
- llvm_i32_ty, // idx_size
- llvm_i32_ty], // repeat_factor: 0..6 for .x1 .. .x64
+ llvm_i32_ty], // idx_size
!listconcat([IntrNoMem, IntrSpeculatable],
- NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>, ArgIndex<4>>.Prop)>;
+ NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>>.Prop)>;
def int_nvvm_spcompress_sp2to4 : SPCompress;
@@ -4263,10 +4256,9 @@ class SPDecompress :
[llvm_anyint_ty, // mdata
llvm_anyint_ty], // cdata
[llvm_i32_ty, // elem_size
- llvm_i32_ty, // idx_size
- llvm_i32_ty], // repeat_factor: 0..6 for .x1 .. .x64
+ llvm_i32_ty], // idx_size
!listconcat([IntrNoMem, IntrSpeculatable],
- NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>, ArgIndex<4>>.Prop)>;
+ NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>>.Prop)>;
foreach num_src = [1, 2, 4] in
foreach num_tgt = [2, 4, 8, 16] in
diff --git a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
index f1562fe948cd43..be362b2a5a2fe8 100644
--- a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
+++ b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
@@ -226,14 +226,14 @@ enum class SPIdxSize : uint64_t {
B4 = 4,
};
-enum class SPLg2RepeatFactor : uint64_t {
- X1 = 0,
- X2 = 1,
- X4 = 2,
- X8 = 3,
- X16 = 4,
- X32 = 5,
- X64 = 6,
+enum class SPRepeatFactor : uint64_t {
+ X1 = 1,
+ X2 = 2,
+ X4 = 4,
+ X8 = 8,
+ X16 = 16,
+ X32 = 32,
+ X64 = 64,
};
struct SPOperandLayout {
@@ -265,20 +265,21 @@ inline StringRef getSPIdxSizeName(SPIdxSize IdxSize) {
return EnumStrings(SPIdxSizeNames).toString(IdxSize);
}
-inline StringRef getSPRepeatFactorName(SPLg2RepeatFactor Lg2RepeatFactor) {
- static constexpr EnumStringDef<SPLg2RepeatFactor> SPRepeatFactorNameDefs[] = {
- {{".x1"}, SPLg2RepeatFactor::X1}, {{".x2"}, SPLg2RepeatFactor::X2},
- {{".x4"}, SPLg2RepeatFactor::X4}, {{".x8"}, SPLg2RepeatFactor::X8},
- {{".x16"}, SPLg2RepeatFactor::X16}, {{".x32"}, SPLg2RepeatFactor::X32},
- {{".x64"}, SPLg2RepeatFactor::X64},
+inline StringRef getSPRepeatFactorName(SPRepeatFactor RepeatFactor) {
+ static constexpr EnumStringDef<SPRepeatFactor> SPRepeatFactorNameDefs[] = {
+ {{".x1"}, SPRepeatFactor::X1}, {{".x2"}, SPRepeatFactor::X2},
+ {{".x4"}, SPRepeatFactor::X4}, {{".x8"}, SPRepeatFactor::X8},
+ {{".x16"}, SPRepeatFactor::X16}, {{".x32"}, SPRepeatFactor::X32},
+ {{".x64"}, SPRepeatFactor::X64},
};
static constexpr auto SPRepeatFactorNames =
BUILD_ENUM_STRINGS(SPRepeatFactorNameDefs);
- return EnumStrings(SPRepeatFactorNames).toString(Lg2RepeatFactor);
+ return EnumStrings(SPRepeatFactorNames).toString(RepeatFactor);
}
-inline unsigned getSPRepeatFactor(unsigned Lg2RepeatFactor) {
- return 1U << Lg2RepeatFactor;
+inline bool isValidSPRepeatFactor(unsigned RepeatFactor) {
+ return !getSPRepeatFactorName(static_cast<SPRepeatFactor>(RepeatFactor))
+ .empty();
}
inline bool isValidSPDecompressFactor(unsigned NumSrc, unsigned NumTgt) {
@@ -296,14 +297,12 @@ inline bool isValidSPDecompressFactor(unsigned NumSrc, unsigned NumTgt) {
inline std::optional<SPOperandLayout>
getSPCompressLayout(unsigned ElemSize, unsigned IdxSize,
- unsigned Lg2RepeatFactor) {
+ unsigned RepeatFactor) {
if (getSPElemSizeName(static_cast<SPElemSize>(ElemSize)).empty() ||
getSPIdxSizeName(static_cast<SPIdxSize>(IdxSize)).empty() ||
- getSPRepeatFactorName(static_cast<SPLg2RepeatFactor>(Lg2RepeatFactor))
- .empty())
+ !isValidSPRepeatFactor(RepeatFactor))
return std::nullopt;
- unsigned RepeatFactor = getSPRepeatFactor(Lg2RepeatFactor);
SPOperandLayout Layout = {
divideCeil(RepeatFactor * IdxSize, ElemSize),
RepeatFactor,
@@ -317,17 +316,15 @@ getSPCompressLayout(unsigned ElemSize, unsigned IdxSize,
inline std::optional<SPOperandLayout>
getSPDecompressLayout(unsigned NumSrc, unsigned NumTgt, unsigned ElemSize,
- unsigned IdxSize, unsigned Lg2RepeatFactor) {
+ unsigned IdxSize, unsigned RepeatFactor) {
if (!isValidSPDecompressFactor(NumSrc, NumTgt) ||
getSPElemSizeName(static_cast<SPElemSize>(ElemSize)).empty() ||
getSPIdxSizeName(static_cast<SPIdxSize>(IdxSize)).empty() ||
- getSPRepeatFactorName(static_cast<SPLg2RepeatFactor>(Lg2RepeatFactor))
- .empty())
+ !isValidSPRepeatFactor(RepeatFactor))
return std::nullopt;
if (NumSrc * ElemSize > 32 || (IdxSize == 2 && NumTgt > 4))
return std::nullopt;
- unsigned RepeatFactor = getSPRepeatFactor(Lg2RepeatFactor);
unsigned DataBits = NumTgt * ElemSize * RepeatFactor;
if (DataBits < 32 || DataBits > 4096)
return std::nullopt;
@@ -364,8 +361,6 @@ LLVM_ABI void printTcgen05MMACollectorBBuffer(raw_ostream &OS,
LLVM_ABI void printSPElemSize(raw_ostream &OS, const Constant *ImmArgVal);
LLVM_ABI void printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal);
-LLVM_ABI void printSPRepeatFactor(raw_ostream &OS, const Constant *ImmArgVal);
-
LLVM_ABI void printTensormapElemType(raw_ostream &OS,
const Constant *ImmArgVal);
LLVM_ABI void printTensormapInterleaveLayout(raw_ostream &OS,
diff --git a/llvm/lib/IR/NVVMIntrinsicUtils.cpp b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
index 5e176da6878f76..38d94c1a3dff91 100644
--- a/llvm/lib/IR/NVVMIntrinsicUtils.cpp
+++ b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
@@ -146,12 +146,6 @@ void nvvm::printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal) {
OS << getSPIdxSizeName(static_cast<SPIdxSize>(CI->getZExtValue()));
}
-void nvvm::printSPRepeatFactor(raw_ostream &OS, const Constant *ImmArgVal) {
- if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
- OS << getSPRepeatFactorName(
- static_cast<SPLg2RepeatFactor>(CI->getZExtValue()));
-}
-
void nvvm::printTensormapElemType(raw_ostream &OS, const Constant *ImmArgVal) {
static constexpr StringRef TensormapElemTypes[] = {
"u8", "u16", "u32", "s32", "u64", "s64",
diff --git a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
index 44992c36d3930d..42dc737ce7e463 100644
--- a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
+++ b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
@@ -513,7 +513,7 @@ void NVPTXInstPrinter::printSPQualifier(const MCInst *MI, int OpNum,
if (Modifier == "repeat_factor") {
StringRef RepeatFactor =
- nvvm::getSPRepeatFactorName(static_cast<nvvm::SPLg2RepeatFactor>(Imm));
+ nvvm::getSPRepeatFactorName(static_cast<nvvm::SPRepeatFactor>(Imm));
assert(!RepeatFactor.empty() && "invalid SP repeat_factor");
O << RepeatFactor;
return;
@@ -564,9 +564,9 @@ void NVPTXInstPrinter::printSPQualifier(const MCInst *MI, int OpNum,
unsigned NumSrc = MI->getOperand(OpNum + 3).getImm();
unsigned NumTgt = MI->getOperand(OpNum + 4).getImm();
auto Layout = GetSPLayout(
- [&](unsigned ElemSize, unsigned IdxSize, unsigned Lg2RepeatFactor) {
+ [&](unsigned ElemSize, unsigned IdxSize, unsigned RepeatFactor) {
return nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSize, IdxSize,
- Lg2RepeatFactor);
+ RepeatFactor);
});
assert(Layout && "invalid spdecompress qualifiers");
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
index 39196bffd851d8..44c90c9237245b 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
@@ -463,7 +463,6 @@ static void checkSPRegTypes(SDNode *N, unsigned NumFlags, StringRef Name) {
// TableGen-only count reaches llvm_unreachable in the selectors below.
#define SPCOMPRESS_OPCODE_CASES(CASE_MACRO) \
/* .b8.b2 and .b16.b4: ceil(repeat_factor / 4) + repeat_factor. */ \
- /* The repeat_factor operand encodes the exponent. */ \
CASE_MACRO(2) /* .x1 */ \
CASE_MACRO(3) /* .x2 */ \
CASE_MACRO(5) /* .x4 */ \
@@ -494,7 +493,7 @@ static void checkSPRegTypes(SDNode *N, unsigned NumFlags, StringRef Name) {
CASE_MACRO(128)
void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
- constexpr unsigned NumFlags = 3;
+ constexpr unsigned NumFlags = 2;
if (N->getNumOperands() < NumFlags + 2)
report_fatal_error("Malformed spcompress intrinsic");
@@ -503,15 +502,19 @@ void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
unsigned IdxSize =
cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
- unsigned Lg2RepeatFactor =
- cast<ConstantSDNode>(N->getOperand(FlagOp + 2))->getZExtValue();
+ checkSPRegTypes(N, NumFlags, "spcompress");
+
+ // The data bundle contains two registers per repetition.
+ unsigned DataSize = FlagOp - 2;
+ unsigned RepeatFactor = DataSize / 2;
+ if (DataSize % 2 != 0 || !nvvm::isValidSPRepeatFactor(RepeatFactor))
+ report_fatal_error("spcompress operand/result types do not match "
+ "the SP intrinsic flags");
- auto Layout = nvvm::getSPCompressLayout(ElemSize, IdxSize, Lg2RepeatFactor);
+ auto Layout = nvvm::getSPCompressLayout(ElemSize, IdxSize, RepeatFactor);
if (!Layout)
report_fatal_error(Twine("Invalid spcompress flags: elem_size=") +
- Twine(ElemSize) + ", idx_size=" + Twine(IdxSize) +
- ", repeat_factor=" + Twine(Lg2RepeatFactor));
- checkSPRegTypes(N, NumFlags, "spcompress");
+ Twine(ElemSize) + ", idx_size=" + Twine(IdxSize));
if (N->getNumValues() != Layout->MetadataSize + Layout->CompressedDataSize ||
N->getNumOperands() != Layout->DataSize + NumFlags + 2)
@@ -523,7 +526,7 @@ void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
Operands.reserve(N->getNumOperands() - NumFlags + 2);
Operands.push_back(getI32Imm(ElemSize, DL));
Operands.push_back(getI32Imm(IdxSize, DL));
- Operands.push_back(getI32Imm(Lg2RepeatFactor, DL));
+ Operands.push_back(getI32Imm(RepeatFactor, DL));
for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
Operands.push_back(N->getOperand(I));
@@ -544,7 +547,7 @@ void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
unsigned NumTgt) {
- constexpr unsigned NumFlags = 3;
+ constexpr unsigned NumFlags = 2;
if (N->getNumOperands() < NumFlags + 3)
report_fatal_error("Malformed spdecompress intrinsic");
@@ -553,18 +556,27 @@ void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
unsigned IdxSize =
cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
- unsigned Lg2RepeatFactor =
- cast<ConstantSDNode>(N->getOperand(FlagOp + 2))->getZExtValue();
+ checkSPRegTypes(N, NumFlags, "spdecompress");
+
+ // data_size = num_tgt * elem_size * repeat_factor / 32.
+ unsigned RepeatFactorDenominator = NumTgt * ElemSize;
+ unsigned RepeatFactorNumerator = N->getNumValues() * 32;
+ if (RepeatFactorDenominator == 0 ||
+ RepeatFactorNumerator % RepeatFactorDenominator != 0)
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
+ unsigned RepeatFactor = RepeatFactorNumerator / RepeatFactorDenominator;
+ if (!nvvm::isValidSPRepeatFactor(RepeatFactor))
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
auto Layout = nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSize, IdxSize,
- Lg2RepeatFactor);
+ RepeatFactor);
if (!Layout)
report_fatal_error(Twine("Invalid spdecompress flags: num_src=") +
Twine(NumSrc) + ", num_tgt=" + Twine(NumTgt) +
", elem_size=" + Twine(ElemSize) +
- ", idx_size=" + Twine(IdxSize) +
- ", repeat_factor=" + Twine(Lg2RepeatFactor));
- checkSPRegTypes(N, NumFlags, "spdecompress");
+ ", idx_size=" + Twine(IdxSize));
if (N->getNumValues() != Layout->DataSize ||
N->getNumOperands() !=
@@ -577,7 +589,7 @@ void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
Operands.reserve(N->getNumOperands() - NumFlags + 4);
Operands.push_back(getI32Imm(ElemSize, DL));
Operands.push_back(getI32Imm(IdxSize, DL));
- Operands.push_back(getI32Imm(Lg2RepeatFactor, DL));
+ Operands.push_back(getI32Imm(RepeatFactor, DL));
Operands.push_back(getI32Imm(NumSrc, DL));
Operands.push_back(getI32Imm(NumTgt, DL));
for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
diff --git a/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td b/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
index 7e8b012c03a236..4a8361d3036693 100644
--- a/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
+++ b/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
@@ -7710,8 +7710,7 @@ class SPFactorName<int NumSrc, int NumTgt> {
string ret = ".sp::" # NumSrc # ":" # NumTgt;
}
-// mdata + cdata = ceil(repeat_factor * idx_size / elem_size) + repeat_factor,
-// where the repeat_factor operand encodes the exponent.
+// mdata + cdata = ceil(repeat_factor * idx_size / elem_size) + repeat_factor.
// The first group is shared by .b8.b2 and .b16.b4. The next groups are the
// additional unique counts for .b16.b2 and .b8.b4.
defvar SPCOMPRESS_OPCODE_CASES =
@@ -7720,13 +7719,10 @@ defvar SPCOMPRESS_OPCODE_CASES =
[6, 12, 24, 48, 96]),
NumResultRegs);
-// data = ceil(num_tgt * elem_size * repeat_factor / 32), where repeat_factor is
-// encoded as an exponent. The legal spdecompress layouts produce all
-// power-of-two data register counts from 1 to 128.
-defvar SPDECOMPRESS_OPCODE_CASES =
- !listconcat(!foreach(lg2_rpf, !range(0, NVVM_SP_LG2_REPEAT_FACTOR_LIMIT),
- !shl(1, lg2_rpf)),
- [128]);
+// data = ceil(num_tgt * elem_size * repeat_factor / 32). The legal
+// spdecompress layouts produce all power-of-two data register counts from 1 to
+// 128.
+defvar SPDECOMPRESS_OPCODE_CASES = [1, 2, 4, 8, 16, 32, 64, 128];
class SPCOMPRESS_INST<int NumResultRegs, int NumSrc, int NumTgt>
: NVPTXInst<(outs), (ins), "?", []> {
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
index 4f5113f0196e1b..57d326a64b0846 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
@@ -1,7 +1,6 @@
; RUN: split-file %s %t
; RUN: not llvm-as -disable-output < %t/bad-elem-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
-; RUN: not llvm-as -disable-output < %t/bad-repeat-factor.ll 2>&1 | FileCheck %s --check-prefix=BAD-RPF
; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
@@ -13,7 +12,7 @@
define { i32, i32 } @bad_spcompress_elem_size(i32 %spdesc,
<2 x i32> %data) {
; BAD-ELEM: immarg value 9 for arg 2 out of range set
- %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 9, i32 2, i32 0)
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 9, i32 2)
ret { i32, i32 } %res
}
@@ -22,16 +21,7 @@ define { i32, i32 } @bad_spcompress_elem_size(i32 %spdesc,
define { i32, i32 } @bad_spcompress_idx_size(i32 %spdesc,
<2 x i32> %data) {
; BAD-IDX: immarg value 3 for arg 3 out of range set
- %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 3, i32 0)
- ret { i32, i32 } %res
-}
-
-;--- bad-repeat-factor.ll
-
-define { i32, i32 } @bad_spcompress_repeat_factor(i32 %spdesc,
- <2 x i32> %data) {
-; BAD-RPF: immarg value 7 for arg 4 out of range [0,7)
- %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 7)
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 3)
ret { i32, i32 } %res
}
@@ -40,7 +30,7 @@ define { i32, i32 } @bad_spcompress_repeat_factor(i32 %spdesc,
define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
<8 x i32> %data) {
; BAD-MDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4)
%mdata = extractvalue { i32, <4 x i32> } %res, 0
ret i32 %mdata
}
@@ -50,7 +40,7 @@ define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
define <2 x i32> @bad_spcompress_compressed_data_register_count(
i32 %spdesc, <8 x i32> %data) {
; BAD-CDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { <2 x i32>, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v2i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %res = call { <2 x i32>, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v2i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4)
%cdata = extractvalue { <2 x i32>, <2 x i32> } %res, 1
ret <2 x i32> %cdata
}
@@ -58,9 +48,9 @@ define <2 x i32> @bad_spcompress_compressed_data_register_count(
;--- bad-data-size.ll
define <4 x i32> @bad_spcompress_data_register_count(i32 %spdesc,
- <4 x i32> %data) {
+ <6 x i32> %data) {
; BAD-DATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v6i32(<6 x i32> %data, i32 %spdesc, i32 8, i32 4)
%cdata = extractvalue { <2 x i32>, <4 x i32> } %res, 1
ret <4 x i32> %cdata
}
@@ -69,7 +59,7 @@ define <4 x i32> @bad_spcompress_data_register_count(i32 %spdesc,
define i64 @bad_spcompress_result_type(i32 %spdesc, <2 x i32> %data) {
; BAD-RESULT-TY: LLVM ERROR: spcompress expects scalarized i32 results
- %res = call { i64, i32 } @llvm.nvvm.spcompress.sp2to4.i64.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ %res = call { i64, i32 } @llvm.nvvm.spcompress.sp2to4.i64.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2)
%mdata = extractvalue { i64, i32 } %res, 0
ret i64 %mdata
}
@@ -78,6 +68,6 @@ define i64 @bad_spcompress_result_type(i32 %spdesc, <2 x i32> %data) {
define { i32, i32 } @bad_spcompress_data_type(i32 %spdesc, i64 %data) {
; BAD-DATA-TY: LLVM ERROR: spcompress expects scalarized i32 operands
- %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.i64(i64 %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.i64(i64 %data, i32 %spdesc, i32 8, i32 2)
ret { i32, i32 } %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
index cc92575a3bbea6..d12a3d03a5fb10 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
@@ -5,7 +5,7 @@
define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32> %data) {
; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -17,13 +17,13 @@ define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2)
ret {i32, i32} %res
}
define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32> %data) {
; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -35,13 +35,13 @@ define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <2 x i3
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 0)
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 2)
ret {i32, i32} %res
}
define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32> %data) {
; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -53,13 +53,13 @@ define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 0)
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 4)
ret {i32, i32} %res
}
define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32> %data) {
; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -71,13 +71,13 @@ define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <2 x i3
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 0)
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 4)
ret {i32, i32} %res
}
define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <4 x i32> %data) {
; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -89,13 +89,13 @@ define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <4
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 1)
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 2)
ret {i32, <2 x i32>} %res
}
define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <4 x i32> %data) {
; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -107,13 +107,13 @@ define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 1)
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 2)
ret {i32, <2 x i32>} %res
}
define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <4 x i32> %data) {
; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -125,13 +125,13 @@ define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <4
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 1)
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 4)
ret {i32, <2 x i32>} %res
}
define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <4 x i32> %data) {
; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -143,13 +143,13 @@ define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 1)
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 4)
ret {i32, <2 x i32>} %res
}
define {i32, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <8 x i32> %data) {
; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -162,13 +162,13 @@ define {i32, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <8
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 2)
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 2)
ret {i32, <4 x i32>} %res
}
define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <8 x i32> %data) {
; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -181,13 +181,13 @@ define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 2)
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 2)
ret {i32, <4 x i32>} %res
}
define {<2 x i32>, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spdesc, <8 x i32> %data) {
; FORMAT-LABEL: define { <2 x i32>, <4 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<16>;
@@ -200,13 +200,13 @@ define {<2 x i32>, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r12, %r13, %r14, %r15};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r10, %r11};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %res = call {<2 x i32>, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4)
ret {<2 x i32>, <4 x i32>} %res
}
define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <8 x i32> %data) {
; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -219,13 +219,13 @@ define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 2)
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 4)
ret {i32, <4 x i32>} %res
}
define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spdesc, <16 x i32> %data) {
; FORMAT-LABEL: define { <2 x i32>, <8 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<28>;
@@ -241,13 +241,13 @@ define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 3)
+ %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 2)
ret {<2 x i32>, <8 x i32>} %res
}
define {i32, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <16 x i32> %data) {
; FORMAT-LABEL: define { i32, <8 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call { i32, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call { i32, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -263,13 +263,13 @@ define {i32, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r18;
; CHECK-NEXT: ret;
- %res = call {i32, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 3)
+ %res = call {i32, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 2)
ret {i32, <8 x i32>} %res
}
define {<4 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spdesc, <16 x i32> %data) {
; FORMAT-LABEL: define { <4 x i32>, <8 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call { <4 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call { <4 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<30>;
@@ -285,13 +285,13 @@ define {<4 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r22, %r23, %r24, %r25};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r18, %r19, %r20, %r21};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 3)
+ %res = call {<4 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 4)
ret {<4 x i32>, <8 x i32>} %res
}
define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spdesc, <16 x i32> %data) {
; FORMAT-LABEL: define { <2 x i32>, <8 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<28>;
@@ -307,13 +307,13 @@ define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spd
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 3)
+ %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 4)
ret {<2 x i32>, <8 x i32>} %res
}
define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spdesc, <32 x i32> %data) {
; FORMAT-LABEL: define { <4 x i32>, <16 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<54>;
@@ -335,13 +335,13 @@ define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %sp
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 4)
+ %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 2)
ret {<4 x i32>, <16 x i32>} %res
}
define {<2 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %spdesc, <32 x i32> %data) {
; FORMAT-LABEL: define { <2 x i32>, <16 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<52>;
@@ -363,13 +363,13 @@ define {<2 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r36, %r37, %r38, %r39};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r34, %r35};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 4)
+ %res = call {<2 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 2)
ret {<2 x i32>, <16 x i32>} %res
}
define {<8 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spdesc, <32 x i32> %data) {
; FORMAT-LABEL: define { <8 x i32>, <16 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call { <8 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call { <8 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<58>;
@@ -392,13 +392,13 @@ define {<8 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %sp
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 4)
+ %res = call {<8 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 4)
ret {<8 x i32>, <16 x i32>} %res
}
define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %spdesc, <32 x i32> %data) {
; FORMAT-LABEL: define { <4 x i32>, <16 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<54>;
@@ -420,13 +420,13 @@ define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 4)
+ %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 4)
ret {<4 x i32>, <16 x i32>} %res
}
define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %spdesc, <64 x i32> %data) {
; FORMAT-LABEL: define { <8 x i32>, <32 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<106>;
@@ -461,13 +461,13 @@ define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %sp
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 5)
+ %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 2)
ret {<8 x i32>, <32 x i32>} %res
}
define {<4 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %spdesc, <64 x i32> %data) {
; FORMAT-LABEL: define { <4 x i32>, <32 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<102>;
@@ -501,13 +501,13 @@ define {<4 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 5)
+ %res = call {<4 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 2)
ret {<4 x i32>, <32 x i32>} %res
}
define {<16 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %spdesc, <64 x i32> %data) {
; FORMAT-LABEL: define { <16 x i32>, <32 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call { <16 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call { <16 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<114>;
@@ -544,13 +544,13 @@ define {<16 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 5)
+ %res = call {<16 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 4)
ret {<16 x i32>, <32 x i32>} %res
}
define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %spdesc, <64 x i32> %data) {
; FORMAT-LABEL: define { <8 x i32>, <32 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<106>;
@@ -585,13 +585,13 @@ define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 5)
+ %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 4)
ret {<8 x i32>, <32 x i32>} %res
}
define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %spdesc, <128 x i32> %data) {
; FORMAT-LABEL: define { <16 x i32>, <64 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<210>;
@@ -652,13 +652,13 @@ define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 6)
+ %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 2)
ret {<16 x i32>, <64 x i32>} %res
}
define {<8 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %spdesc, <128 x i32> %data) {
; FORMAT-LABEL: define { <8 x i32>, <64 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<202>;
@@ -717,13 +717,13 @@ define {<8 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 6)
+ %res = call {<8 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 2)
ret {<8 x i32>, <64 x i32>} %res
}
define {<32 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %spdesc, <128 x i32> %data) {
; FORMAT-LABEL: define { <32 x i32>, <64 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call { <32 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call { <32 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<226>;
@@ -788,13 +788,13 @@ define {<32 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<32 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 6)
+ %res = call {<32 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 4)
ret {<32 x i32>, <64 x i32>} %res
}
define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %spdesc, <128 x i32> %data) {
; FORMAT-LABEL: define { <16 x i32>, <64 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<210>;
@@ -855,6 +855,6 @@ define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 6)
+ %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 4)
ret {<16 x i32>, <64 x i32>} %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
index 74d6eaa53dc66d..18e9e5cfa460f1 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
@@ -1,10 +1,8 @@
; RUN: split-file %s %t
; RUN: not llvm-as -disable-output < %t/bad-elem-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
-; RUN: not llvm-as -disable-output < %t/bad-repeat-factor.ll 2>&1 | FileCheck %s --check-prefix=BAD-RPF
; RUN: not --crash llc < %t/bad-idx-size-for-target.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-IDX-TARGET
; RUN: not --crash llc < %t/bad-input-span.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-INPUT-SPAN
-; RUN: not --crash llc < %t/bad-data-too-small.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA-SMALL
; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
@@ -15,7 +13,7 @@
define i32 @bad_spdecompress_elem_size(i32 %metadata, i32 %compdata) {
; BAD-ELEM: immarg value 9 for arg 2 out of range set
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 9, i32 2, i32 0)
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 9, i32 2)
ret i32 %res
}
@@ -23,15 +21,7 @@ define i32 @bad_spdecompress_elem_size(i32 %metadata, i32 %compdata) {
define i32 @bad_spdecompress_idx_size(i32 %metadata, i32 %compdata) {
; BAD-IDX: immarg value 3 for arg 3 out of range set
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 3, i32 0)
- ret i32 %res
-}
-
-;--- bad-repeat-factor.ll
-
-define i32 @bad_spdecompress_repeat_factor(i32 %metadata, i32 %compdata) {
-; BAD-RPF: immarg value 7 for arg 4 out of range [0,7)
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 7)
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 3)
ret i32 %res
}
@@ -39,8 +29,8 @@ define i32 @bad_spdecompress_repeat_factor(i32 %metadata, i32 %compdata) {
define <8 x i32> @bad_spdecompress_idx_size_for_target(
i32 %metadata, i32 %compdata) {
-; BAD-IDX-TARGET: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=8, elem_size=8, idx_size=2, repeat_factor=2
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+; BAD-IDX-TARGET: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=8, elem_size=8, idx_size=2
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret <8 x i32> %res
}
@@ -48,25 +38,17 @@ define <8 x i32> @bad_spdecompress_idx_size_for_target(
define <4 x i32> @bad_spdecompress_input_span(<2 x i32> %metadata,
<2 x i32> %compdata) {
-; BAD-INPUT-SPAN: LLVM ERROR: Invalid spdecompress flags: num_src=4, num_tgt=8, elem_size=16, idx_size=4, repeat_factor=0
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 0)
+; BAD-INPUT-SPAN: LLVM ERROR: Invalid spdecompress flags: num_src=4, num_tgt=8, elem_size=16, idx_size=4
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 16, i32 4)
ret <4 x i32> %res
}
-;--- bad-data-too-small.ll
-
-define i32 @bad_spdecompress_data_too_small(i32 %metadata, i32 %compdata) {
-; BAD-DATA-SMALL: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=2, elem_size=8, idx_size=2, repeat_factor=0
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
- ret i32 %res
-}
-
;--- bad-mdata-size.ll
define <4 x i32> @bad_spdecompress_metadata_register_count(
<2 x i32> %metadata, <2 x i32> %compdata) {
; BAD-MDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
@@ -75,24 +57,24 @@ define <4 x i32> @bad_spdecompress_metadata_register_count(
define <4 x i32> @bad_spdecompress_compressed_data_register_count(
i32 %metadata, i32 %compdata) {
; BAD-CDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <4 x i32> %res
}
;--- bad-data-size.ll
-define <2 x i32> @bad_spdecompress_data_register_count(
+define <3 x i32> @bad_spdecompress_data_register_count(
i32 %metadata, <2 x i32> %compdata) {
; BAD-DATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
- ret <2 x i32> %res
+ %res = call <3 x i32> @llvm.nvvm.spdecompress.sp2to4.v3i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
+ ret <3 x i32> %res
}
;--- bad-result-type.ll
define i64 @bad_spdecompress_result_type(i32 %metadata, i32 %compdata) {
; BAD-RESULT-TY: LLVM ERROR: spdecompress expects scalarized i32 results
- %res = call i64 @llvm.nvvm.spdecompress.sp2to4.i64.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ %res = call i64 @llvm.nvvm.spdecompress.sp2to4.i64.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret i64 %res
}
@@ -100,6 +82,6 @@ define i64 @bad_spdecompress_result_type(i32 %metadata, i32 %compdata) {
define i32 @bad_spdecompress_metadata_type(i64 %metadata, i32 %compdata) {
; BAD-MDATA-TY: LLVM ERROR: spdecompress expects scalarized i32 operands
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i64.i32(i64 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i64.i32(i64 %metadata, i32 %compdata, i32 8, i32 2)
ret i32 %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
index e375269b36e494..52abb3c66a937d 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4
define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -17,13 +17,13 @@ define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -35,13 +35,13 @@ define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<19>;
@@ -55,13 +55,13 @@ define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<36>;
@@ -79,13 +79,13 @@ define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<71>;
@@ -111,13 +111,13 @@ define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <64 x i32> %res
}
define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<141>;
@@ -160,14 +160,14 @@ define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <128 x i32> %res
}
; elem_size=16, idx_size=4
define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -179,13 +179,13 @@ define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<19>;
@@ -199,13 +199,13 @@ define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<36>;
@@ -223,13 +223,13 @@ define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<70>;
@@ -255,13 +255,13 @@ define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
ret <64 x i32> %res
}
define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<139>;
@@ -304,6 +304,6 @@ define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
ret <128 x i32> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
index e90aed6e35fb07..8a7260f878643d 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
@@ -5,7 +5,7 @@
define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -16,13 +16,13 @@ define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x2 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret i32 %res
}
define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -33,13 +33,13 @@ define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -50,13 +50,13 @@ define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 3)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -68,13 +68,13 @@ define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 4)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -89,13 +89,13 @@ define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 5)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -116,13 +116,13 @@ define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 6)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2)
ret <32 x i32> %res
}
define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -133,13 +133,13 @@ define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x2 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret i32 %res
}
define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -150,13 +150,13 @@ define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -167,13 +167,13 @@ define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -185,13 +185,13 @@ define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -206,13 +206,13 @@ define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -234,13 +234,13 @@ define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -251,13 +251,13 @@ define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
ret i32 %res
}
define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -268,13 +268,13 @@ define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 1)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -285,13 +285,13 @@ define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 2)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -303,13 +303,13 @@ define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 3)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<26>;
@@ -324,13 +324,13 @@ define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 4)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<51>;
@@ -351,13 +351,13 @@ define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 5)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<101>;
@@ -390,13 +390,13 @@ define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 6)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2)
ret <64 x i32> %res
}
define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -407,13 +407,13 @@ define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret i32 %res
}
define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -424,13 +424,13 @@ define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -441,13 +441,13 @@ define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -459,13 +459,13 @@ define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -480,13 +480,13 @@ define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -507,13 +507,13 @@ define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -547,6 +547,6 @@ define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 6)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4)
ret <64 x i32> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
index c9c369e724a074..b30695cba90351 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=2
define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -17,13 +17,13 @@ define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret i32 %res
}
define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -34,13 +34,13 @@ define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -51,13 +51,13 @@ define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -69,13 +69,13 @@ define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 3)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -89,13 +89,13 @@ define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 4)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -114,13 +114,13 @@ define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 5)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -149,14 +149,14 @@ define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 6)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2)
ret <64 x i32> %res
}
; elem_size=8, idx_size=4
define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -167,13 +167,13 @@ define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret i32 %res
}
define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -184,13 +184,13 @@ define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -201,13 +201,13 @@ define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -219,13 +219,13 @@ define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -239,13 +239,13 @@ define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -264,13 +264,13 @@ define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -300,14 +300,14 @@ define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
ret <64 x i32> %res
}
; elem_size=16, idx_size=2
define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -318,13 +318,13 @@ define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -335,13 +335,13 @@ define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 1)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -353,13 +353,13 @@ define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 2)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -373,13 +373,13 @@ define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 3)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<42>;
@@ -398,13 +398,13 @@ define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 4)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<83>;
@@ -433,13 +433,13 @@ define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 5)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2)
ret <64 x i32> %res
}
define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<165>;
@@ -488,14 +488,14 @@ define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 6)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2)
ret <128 x i32> %res
}
; elem_size=16, idx_size=4
define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -506,13 +506,13 @@ define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -523,13 +523,13 @@ define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -541,13 +541,13 @@ define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -561,13 +561,13 @@ define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -586,13 +586,13 @@ define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -621,13 +621,13 @@ define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
ret <64 x i32> %res
}
define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<169>;
@@ -677,6 +677,6 @@ define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 6)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4)
ret <128 x i32> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
index c9c5347fa485e6..f98b15bc84c92a 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4
define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -17,13 +17,13 @@ define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <2 x i32> %res
}
define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -34,13 +34,13 @@ define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -52,13 +52,13 @@ define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -72,13 +72,13 @@ define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<39>;
@@ -96,13 +96,13 @@ define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<77>;
@@ -129,13 +129,13 @@ define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <64 x i32> %res
}
define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<153>;
@@ -181,14 +181,14 @@ define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
ret <128 x i32> %res
}
; elem_size=16, idx_size=4
define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -199,13 +199,13 @@ define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <4 x i32> %res
}
define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -217,13 +217,13 @@ define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <8 x i32> %res
}
define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -237,13 +237,13 @@ define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
ret <16 x i32> %res
}
define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<38>;
@@ -261,13 +261,13 @@ define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
ret <32 x i32> %res
}
define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<75>;
@@ -294,13 +294,13 @@ define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
ret <64 x i32> %res
}
define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<149>;
@@ -345,6 +345,6 @@ define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
ret <128 x i32> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
index a85bda8ebb5e47..7b3e1d8261aa89 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -17,14 +17,14 @@ define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
; elem_size=8, idx_size=4, rpf=2
define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -36,14 +36,14 @@ define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
; elem_size=8, idx_size=4, rpf=4
define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -57,14 +57,14 @@ define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
; elem_size=8, idx_size=4, rpf=8
define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<39>;
@@ -82,14 +82,14 @@ define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
; elem_size=8, idx_size=4, rpf=16
define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<77>;
@@ -116,14 +116,14 @@ define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <64 x i32> %res
}
; elem_size=8, idx_size=4, rpf=32
define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<153>;
@@ -169,14 +169,14 @@ define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
ret <128 x i32> %res
}
; elem_size=16, idx_size=4, rpf=1
define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -188,14 +188,14 @@ define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <8 x i32> %res
}
; elem_size=16, idx_size=4, rpf=2
define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -209,14 +209,14 @@ define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
ret <16 x i32> %res
}
; elem_size=16, idx_size=4, rpf=4
define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<38>;
@@ -234,14 +234,14 @@ define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
ret <32 x i32> %res
}
; elem_size=16, idx_size=4, rpf=8
define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<75>;
@@ -268,14 +268,14 @@ define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
ret <64 x i32> %res
}
; elem_size=16, idx_size=4, rpf=16
define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<149>;
@@ -320,6 +320,6 @@ define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
ret <128 x i32> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
index d332a2e5fc790e..be01b26892052e 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=2, rpf=1
define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -17,14 +17,14 @@ define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret i32 %res
}
; elem_size=8, idx_size=2, rpf=2
define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -35,14 +35,14 @@ define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
ret <2 x i32> %res
}
; elem_size=8, idx_size=2, rpf=4
define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -53,14 +53,14 @@ define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 2)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2)
ret <4 x i32> %res
}
; elem_size=8, idx_size=2, rpf=8
define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -72,14 +72,14 @@ define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 3)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2)
ret <8 x i32> %res
}
; elem_size=8, idx_size=2, rpf=16
define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -94,14 +94,14 @@ define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 4)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2)
ret <16 x i32> %res
}
; elem_size=8, idx_size=2, rpf=32
define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -122,14 +122,14 @@ define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 5)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2)
ret <32 x i32> %res
}
; elem_size=8, idx_size=2, rpf=64
define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -163,14 +163,14 @@ define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 2, i32 6)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 2)
ret <64 x i32> %res
}
; elem_size=8, idx_size=4, rpf=1
define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -181,14 +181,14 @@ define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret i32 %res
}
; elem_size=8, idx_size=4, rpf=2
define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -199,14 +199,14 @@ define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <2 x i32> %res
}
; elem_size=8, idx_size=4, rpf=4
define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -217,14 +217,14 @@ define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
; elem_size=8, idx_size=4, rpf=8
define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -236,14 +236,14 @@ define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %me
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
; elem_size=8, idx_size=4, rpf=16
define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -258,14 +258,14 @@ define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
; elem_size=8, idx_size=4, rpf=32
define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -287,14 +287,14 @@ define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
; elem_size=8, idx_size=4, rpf=64
define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<113>;
@@ -330,14 +330,14 @@ define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 6)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4)
ret <64 x i32> %res
}
; elem_size=16, idx_size=2, rpf=1
define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -348,14 +348,14 @@ define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
ret <2 x i32> %res
}
; elem_size=16, idx_size=2, rpf=2
define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -366,14 +366,14 @@ define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 1)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2)
ret <4 x i32> %res
}
; elem_size=16, idx_size=2, rpf=4
define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -385,14 +385,14 @@ define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 2)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2)
ret <8 x i32> %res
}
; elem_size=16, idx_size=2, rpf=8
define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<26>;
@@ -407,14 +407,14 @@ define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 3)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2)
ret <16 x i32> %res
}
; elem_size=16, idx_size=2, rpf=16
define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<51>;
@@ -435,14 +435,14 @@ define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 4)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2)
ret <32 x i32> %res
}
; elem_size=16, idx_size=2, rpf=32
define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<101>;
@@ -475,14 +475,14 @@ define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 5)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2)
ret <64 x i32> %res
}
; elem_size=16, idx_size=2, rpf=64
define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32> %metadata, <64 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<201>;
@@ -540,14 +540,14 @@ define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r77, %r78, %r79, %r80};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r73, %r74, %r75, %r76};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 2, i32 6)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 2)
ret <128 x i32> %res
}
; elem_size=16, idx_size=4, rpf=1
define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -558,14 +558,14 @@ define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <2 x i32> %res
}
; elem_size=16, idx_size=4, rpf=2
define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -576,14 +576,14 @@ define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
ret <4 x i32> %res
}
; elem_size=16, idx_size=4, rpf=4
define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -595,14 +595,14 @@ define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
ret <8 x i32> %res
}
; elem_size=16, idx_size=4, rpf=8
define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -617,14 +617,14 @@ define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
ret <16 x i32> %res
}
; elem_size=16, idx_size=4, rpf=16
define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -645,14 +645,14 @@ define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
ret <32 x i32> %res
}
; elem_size=16, idx_size=4, rpf=32
define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -686,14 +686,14 @@ define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 5)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4)
ret <64 x i32> %res
}
; elem_size=16, idx_size=4, rpf=64
define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32> %metadata, <64 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<209>;
@@ -753,6 +753,6 @@ define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r85, %r86, %r87, %r88};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r81, %r82, %r83, %r84};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 4, i32 6)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 4)
ret <128 x i32> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
index 452efc202134fd..206ce5a560697d 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -17,14 +17,14 @@ define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <2 x i32> %res
}
; elem_size=8, idx_size=4, rpf=2
define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -35,14 +35,14 @@ define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
; elem_size=8, idx_size=4, rpf=4
define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -54,14 +54,14 @@ define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
; elem_size=8, idx_size=4, rpf=8
define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -75,14 +75,14 @@ define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
; elem_size=8, idx_size=4, rpf=16
define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -101,14 +101,14 @@ define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
; elem_size=8, idx_size=4, rpf=32
define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -138,14 +138,14 @@ define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
ret <64 x i32> %res
}
; elem_size=8, idx_size=4, rpf=64
define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<177>;
@@ -197,14 +197,14 @@ define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 6)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4)
ret <128 x i32> %res
}
; elem_size=16, idx_size=4, rpf=1
define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -215,14 +215,14 @@ define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
ret <4 x i32> %res
}
; elem_size=16, idx_size=4, rpf=2
define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -234,14 +234,14 @@ define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
ret <8 x i32> %res
}
; elem_size=16, idx_size=4, rpf=4
define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -255,14 +255,14 @@ define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
ret <16 x i32> %res
}
; elem_size=16, idx_size=4, rpf=8
define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -281,14 +281,14 @@ define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
ret <32 x i32> %res
}
; elem_size=16, idx_size=4, rpf=16
define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -317,14 +317,14 @@ define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
ret <64 x i32> %res
}
; elem_size=16, idx_size=4, rpf=32
define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<169>;
@@ -374,6 +374,6 @@ define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 5)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4)
ret <128 x i32> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
index ad2f926a520859..3b8d80c20d8643 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -17,14 +17,14 @@ define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
; elem_size=8, idx_size=4, rpf=2
define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -36,14 +36,14 @@ define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 1)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
; elem_size=8, idx_size=4, rpf=4
define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -57,14 +57,14 @@ define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 2)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
; elem_size=8, idx_size=4, rpf=8
define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -83,14 +83,14 @@ define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
; elem_size=8, idx_size=4, rpf=16
define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -120,14 +120,14 @@ define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
ret <64 x i32> %res
}
; elem_size=8, idx_size=4, rpf=32
define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<177>;
@@ -179,6 +179,6 @@ define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4)
ret <128 x i32> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
index 94bff5c02f6f2a..c77a43a5362513 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -17,14 +17,14 @@ define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
ret <2 x i32> %res
}
; elem_size=8, idx_size=4, rpf=2
define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -35,14 +35,14 @@ define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 1)
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
ret <4 x i32> %res
}
; elem_size=8, idx_size=4, rpf=4
define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <4 x i32> %compdata) {
; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -54,14 +54,14 @@ define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %me
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 2)
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
ret <8 x i32> %res
}
; elem_size=8, idx_size=4, rpf=8
define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <8 x i32> %compdata) {
; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -76,14 +76,14 @@ define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 3)
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
ret <16 x i32> %res
}
; elem_size=8, idx_size=4, rpf=16
define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <16 x i32> %compdata) {
; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -105,14 +105,14 @@ define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 4)
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
ret <32 x i32> %res
}
; elem_size=8, idx_size=4, rpf=32
define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <32 x i32> %compdata) {
; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<113>;
@@ -148,14 +148,14 @@ define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 5)
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4)
ret <64 x i32> %res
}
; elem_size=8, idx_size=4, rpf=64
define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32> %metadata, <64 x i32> %compdata) {
; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<225>;
@@ -219,6 +219,6 @@ define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r101, %r102, %r103, %r104};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r97, %r98, %r99, %r100};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, i32 8, i32 4, i32 6)
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, i32 8, i32 4)
ret <128 x i32> %res
}
>From 6a68516a2b7232ee5d59779824ab64cdcfb80993 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Wed, 9 Sep 2026 21:25:34 +0200
Subject: [PATCH 03/10] [NVVM]][NVPTX] Infer sparse element size from data
types
---
llvm/docs/NVPTXUsage.md | 68 +++--
llvm/include/llvm/IR/IntrinsicsNVVM.td | 25 +-
llvm/include/llvm/IR/NVVMIntrinsicUtils.h | 1 -
llvm/lib/IR/NVVMIntrinsicUtils.cpp | 5 -
llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp | 80 +++--
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp | 147 +++++----
llvm/test/CodeGen/NVPTX/spcompress-invalid.ll | 67 ++---
llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll | 280 +++++++++---------
.../CodeGen/NVPTX/spdecompress-invalid.ll | 80 ++---
.../CodeGen/NVPTX/spdecompress-sp1to16.ll | 110 +++----
.../test/CodeGen/NVPTX/spdecompress-sp1to2.ll | 260 ++++++++--------
.../test/CodeGen/NVPTX/spdecompress-sp1to4.ll | 280 +++++++++---------
.../test/CodeGen/NVPTX/spdecompress-sp1to8.ll | 130 ++++----
.../CodeGen/NVPTX/spdecompress-sp2to16.ll | 110 +++----
.../test/CodeGen/NVPTX/spdecompress-sp2to4.ll | 280 +++++++++---------
.../test/CodeGen/NVPTX/spdecompress-sp2to8.ll | 130 ++++----
.../CodeGen/NVPTX/spdecompress-sp4to16.ll | 60 ++--
.../test/CodeGen/NVPTX/spdecompress-sp4to8.ll | 70 ++---
18 files changed, 1124 insertions(+), 1059 deletions(-)
diff --git a/llvm/docs/NVPTXUsage.md b/llvm/docs/NVPTXUsage.md
index 31b140b722e69a..163886a0436bfc 100644
--- a/llvm/docs/NVPTXUsage.md
+++ b/llvm/docs/NVPTXUsage.md
@@ -1035,19 +1035,22 @@ the PTX `spcompress` and `spdecompress` instructions. They require PTX ISA 9.4
and `sm_107a`.
The intrinsic types representing `mdata`, `cdata`, and `data` are overloaded.
-A bundle of one 32-bit register is represented by `i32`, and a bundle of `N`
-registers is represented by `<N x i32>`. The actual intrinsic names include
-the corresponding LLVM overload suffixes.
+The packed metadata remains a bundle of 32-bit registers: one register is
+represented by `i32`, and `N` registers are represented by `<N x i32>`. The
+`cdata` and `data` bundles instead encode the element size in their scalar type.
+Each 32-bit register is represented by four `i8` lanes or two `i16` lanes, so a
+bundle of `N` registers uses `<4N x i8>` or `<2N x i16>`, respectively. The
+actual intrinsic names include the corresponding LLVM overload suffixes.
-The two trailing arguments are immediate qualifiers with the following
-encodings:
+The trailing argument is an immediate qualifier with the following encoding:
-| Argument | Values | PTX qualifiers |
-| ----------------------- | ------------------------- | -------------------------------------- |
-| `%elem_size` | `8`, `16` | `.b8`, `.b16` |
-| `%idx_size` | `2`, `4` | `.b2`, `.b4` |
+| Argument | Values | PTX qualifiers |
+| ----------- | -------- | -------------- |
+| `%idx_size` | `2`, `4` | `.b2`, `.b4` |
-The PTX repeat-factor qualifier is inferred from the overloaded bundle types.
+The PTX element-size qualifier (`.b8` or `.b16`) is inferred from the `i8` or
+`i16` element type. The repeat-factor qualifier is inferred from the bundle
+sizes.
#### '`llvm.nvvm.spcompress.sp2to4`' Intrinsic
@@ -1055,7 +1058,7 @@ The PTX repeat-factor qualifier is inferred from the overloaded bundle types.
```llvm
declare {MDataTy, CDataTy} @llvm.nvvm.spcompress.sp2to4(
- DataTy %data, i32 %spdesc, i32 immarg %elem_size, i32 immarg %idx_size)
+ DataTy %data, i32 %spdesc, i32 immarg %idx_size)
```
##### Overview:
@@ -1064,14 +1067,16 @@ This intrinsic compresses the dense vector `%data` using 2:4 structured
sparsity. It returns the selected element indices as `mdata` and the selected
elements as `cdata`. The number of 32-bit registers in each bundle is:
-| Bundle | Register count |
-| ------- | ------------------------------------------- |
-| `data` | `2 * num` |
-| `cdata` | `num` |
-| `mdata` | `ceil(num * %idx_size / %elem_size)` |
+| Bundle | Register count |
+| ------- | ----------------------------------------- |
+| `data` | `2 * num` |
+| `cdata` | `num` |
+| `mdata` | `ceil(num * %idx_size / elem_size)` |
-Here, `num` is half the number of registers in `DataTy` and determines the PTX
-repeat-factor qualifier (`.x1`, `.x2`, ..., `.x64`).
+Here, `elem_size` is the scalar bit width of `DataTy` and `CDataTy`. Both types
+must use the same `i8` or `i16` element type. `num` is half the number of
+registers in `DataTy` and determines the PTX repeat-factor qualifier (`.x1`,
+`.x2`, ..., `.x64`).
The combined `mdata`, `cdata`, and `data` bundle size must not exceed 253
registers.
@@ -1085,8 +1090,9 @@ type:
| 2-4 | Element data type | `0`: f16/u8, `1`: bf16/s8, `2`: e5m2, `3`: e4m3, `4`: e3m2, `5`: e2m3 |
| 5-31 | Reserved | `0` |
-The element data type must be consistent with `%elem_size`. When the metadata
-occupies less than 32 bits, it is zero-extended to fill its `i32` register.
+The element data type selected by `%spdesc` must be consistent with the scalar
+element type of `DataTy` and `CDataTy`. When the metadata occupies less than 32
+bits, it is zero-extended to fill its `i32` register.
The operation treats negative zero as less than positive zero. NaN elements
are always selected; when multiple selections satisfy the same comparison,
the selected indices are implementation-specific.
@@ -1102,7 +1108,7 @@ For more information, see the
declare DataTy @llvm.nvvm.spdecompress.{sp1to2,sp1to4,sp1to8,sp1to16,
sp2to4,sp2to8,sp2to16,
sp4to8,sp4to16}(
- MDataTy %mdata, CDataTy %cdata, i32 immarg %elem_size, i32 immarg %idx_size)
+ MDataTy %mdata, CDataTy %cdata, i32 immarg %idx_size)
```
##### Overview:
@@ -1116,20 +1122,22 @@ Dense positions not selected by the metadata are set to zero.
For an intrinsic named `spXtoY`, the number of 32-bit registers in each bundle
is:
-| Bundle | Register count |
-| ------- | --------------------------------------------------- |
-| `mdata` | `ceil(X * %idx_size * num / 32)` |
-| `cdata` | `ceil(X * %elem_size * num / 32)` |
-| `data` | `ceil(Y * %elem_size * num / 32)` |
+| Bundle | Register count |
+| ------- | ------------------------------------------------- |
+| `mdata` | `ceil(X * %idx_size * num / 32)` |
+| `cdata` | `ceil(X * elem_size * num / 32)` |
+| `data` | `ceil(Y * elem_size * num / 32)` |
-Here, `num` is inferred from the number of registers in `DataTy` and determines
-the PTX repeat-factor qualifier (`.x1`, `.x2`, ..., `.x64`).
+Here, `elem_size` is the common `i8` or `i16` scalar bit width of `CDataTy` and
+`DataTy`. `num` is inferred from the number of registers in `DataTy` and
+determines the PTX repeat-factor qualifier (`.x1`, `.x2`, ..., `.x64`). Any
+unused lanes in the final `CDataTy` or `DataTy` register are padding.
The following conditions must hold:
-- `X * %elem_size <= 32`.
+- `X * elem_size <= 32`.
- `%idx_size` is `2` only when `Y <= 4`.
-- `32 <= Y * %elem_size * num <= 4096`.
+- `32 <= Y * elem_size * num <= 4096`.
- The combined `mdata`, `cdata`, and `data` bundle size does not exceed 253
registers.
diff --git a/llvm/include/llvm/IR/IntrinsicsNVVM.td b/llvm/include/llvm/IR/IntrinsicsNVVM.td
index 66948a121ffb40..45d117586d811e 100644
--- a/llvm/include/llvm/IR/IntrinsicsNVVM.td
+++ b/llvm/include/llvm/IR/IntrinsicsNVVM.td
@@ -1335,12 +1335,9 @@ class NVVMPureIntrinsic<list<LLVMType> ret_types,
DefaultAttrsIntrinsic<ret_types, param_types,
intr_properties # [IntrNoMem, IntrSpeculatable], name> {}
-class NVVM_SP_ARGPROPS<ArgIndex ElemSizeIdx, ArgIndex IdxSizeIdx> {
+class NVVM_SP_IDX_ARGPROPS<ArgIndex IdxSizeIdx> {
list<IntrinsicProperty> Prop = [
- RangeList<ElemSizeIdx, [8, 16]>,
RangeList<IdxSizeIdx, [2, 4]>,
- ArgInfo<ElemSizeIdx, [ArgName<"elem_size">,
- ImmArgPrinter<"printSPElemSize">]>,
ArgInfo<IdxSizeIdx, [ArgName<"idx_size">,
ImmArgPrinter<"printSPIdxSize">]>
];
@@ -4236,13 +4233,12 @@ def int_nvvm_tensormap_replace_fill_mode :
class SPCompress :
DefaultAttrsIntrinsicFlags<
[llvm_anyint_ty, // mdata
- llvm_anyint_ty], // cdata
- [llvm_anyint_ty, // data
- llvm_i32_ty], // spdesc
- [llvm_i32_ty, // elem_size
- llvm_i32_ty], // idx_size
+ llvm_any_vector_int_ty], // cdata
+ [llvm_any_vector_int_ty, // data
+ llvm_i32_ty], // spdesc
+ [llvm_i32_ty], // idx_size
!listconcat([IntrNoMem, IntrSpeculatable],
- NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>>.Prop)>;
+ NVVM_SP_IDX_ARGPROPS<ArgIndex<2>>.Prop)>;
def int_nvvm_spcompress_sp2to4 : SPCompress;
@@ -4252,13 +4248,12 @@ def int_nvvm_spcompress_sp2to4 : SPCompress;
class SPDecompress :
DefaultAttrsIntrinsicFlags<
- [llvm_anyint_ty], // data
+ [llvm_any_vector_int_ty], // data
[llvm_anyint_ty, // mdata
- llvm_anyint_ty], // cdata
- [llvm_i32_ty, // elem_size
- llvm_i32_ty], // idx_size
+ llvm_any_vector_int_ty], // cdata
+ [llvm_i32_ty], // idx_size
!listconcat([IntrNoMem, IntrSpeculatable],
- NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>>.Prop)>;
+ NVVM_SP_IDX_ARGPROPS<ArgIndex<2>>.Prop)>;
foreach num_src = [1, 2, 4] in
foreach num_tgt = [2, 4, 8, 16] in
diff --git a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
index be362b2a5a2fe8..17ca42b3ea99a5 100644
--- a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
+++ b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
@@ -359,7 +359,6 @@ LLVM_ABI void printTcgen05CollectorUsageOp(raw_ostream &OS,
LLVM_ABI void printTcgen05MMACollectorBBuffer(raw_ostream &OS,
const Constant *ImmArgVal);
-LLVM_ABI void printSPElemSize(raw_ostream &OS, const Constant *ImmArgVal);
LLVM_ABI void printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal);
LLVM_ABI void printTensormapElemType(raw_ostream &OS,
const Constant *ImmArgVal);
diff --git a/llvm/lib/IR/NVVMIntrinsicUtils.cpp b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
index 38d94c1a3dff91..933ab39104cb14 100644
--- a/llvm/lib/IR/NVVMIntrinsicUtils.cpp
+++ b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
@@ -136,11 +136,6 @@ void nvvm::printTcgen05MMACollectorBBuffer(raw_ostream &OS,
}
}
-void nvvm::printSPElemSize(raw_ostream &OS, const Constant *ImmArgVal) {
- if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
- OS << getSPElemSizeName(static_cast<SPElemSize>(CI->getZExtValue()));
-}
-
void nvvm::printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal) {
if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
OS << getSPIdxSizeName(static_cast<SPIdxSize>(CI->getZExtValue()));
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
index 44c90c9237245b..966ad3c171fabe 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
@@ -448,14 +448,10 @@ void NVPTXDAGToDAGISel::SelectTcgen05Ld(SDNode *N, bool hasOffset) {
}
}
-static void checkSPRegTypes(SDNode *N, unsigned NumFlags, StringRef Name) {
- for (const EVT VT : N->values())
- if (VT != MVT::i32)
- report_fatal_error(Twine(Name) + " expects scalarized i32 results");
-
- for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
- if (N->getOperand(I).getValueType() != MVT::i32)
- report_fatal_error(Twine(Name) + " expects scalarized i32 operands");
+static std::optional<unsigned> getSPDataElemSize(EVT VT) {
+ if (VT != MVT::v4i8 && VT != MVT::v2i16)
+ return std::nullopt;
+ return static_cast<unsigned>(VT.getScalarSizeInBits());
}
// Keep these lists in sync with their counterparts in NVPTXIntrinsics.td. A
@@ -493,38 +489,55 @@ static void checkSPRegTypes(SDNode *N, unsigned NumFlags, StringRef Name) {
CASE_MACRO(128)
void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
- constexpr unsigned NumFlags = 2;
- if (N->getNumOperands() < NumFlags + 2)
+ constexpr unsigned NumFlags = 1;
+ if (N->getNumOperands() < NumFlags + 3)
report_fatal_error("Malformed spcompress intrinsic");
unsigned FlagOp = N->getNumOperands() - NumFlags;
- unsigned ElemSize =
- cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
unsigned IdxSize =
- cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
- checkSPRegTypes(N, NumFlags, "spcompress");
+ cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
// The data bundle contains two registers per repetition.
unsigned DataSize = FlagOp - 2;
unsigned RepeatFactor = DataSize / 2;
- if (DataSize % 2 != 0 || !nvvm::isValidSPRepeatFactor(RepeatFactor))
+ EVT DataRegVT = N->getOperand(1).getValueType();
+ auto ElemSize = getSPDataElemSize(DataRegVT);
+ if (!ElemSize || DataSize % 2 != 0 ||
+ !nvvm::isValidSPRepeatFactor(RepeatFactor))
report_fatal_error("spcompress operand/result types do not match "
"the SP intrinsic flags");
+ unsigned ElemSizeValue = *ElemSize;
- auto Layout = nvvm::getSPCompressLayout(ElemSize, IdxSize, RepeatFactor);
+ for (unsigned I = 1; I <= DataSize; ++I)
+ if (N->getOperand(I).getValueType() != DataRegVT)
+ report_fatal_error("spcompress operand/result types do not match "
+ "the SP intrinsic flags");
+ if (N->getOperand(FlagOp - 1).getValueType() != MVT::i32)
+ report_fatal_error("spcompress operand/result types do not match "
+ "the SP intrinsic flags");
+
+ auto Layout = nvvm::getSPCompressLayout(ElemSizeValue, IdxSize, RepeatFactor);
if (!Layout)
report_fatal_error(Twine("Invalid spcompress flags: elem_size=") +
- Twine(ElemSize) + ", idx_size=" + Twine(IdxSize));
+ Twine(ElemSizeValue) + ", idx_size=" + Twine(IdxSize));
if (N->getNumValues() != Layout->MetadataSize + Layout->CompressedDataSize ||
N->getNumOperands() != Layout->DataSize + NumFlags + 2)
report_fatal_error("spcompress operand/result types do not match "
"the SP intrinsic flags");
+ for (unsigned I = 0; I != Layout->MetadataSize; ++I)
+ if (N->getValueType(I) != MVT::i32)
+ report_fatal_error("spcompress operand/result types do not match "
+ "the SP intrinsic flags");
+ for (unsigned I = Layout->MetadataSize, E = N->getNumValues(); I != E; ++I)
+ if (N->getValueType(I) != DataRegVT)
+ report_fatal_error("spcompress operand/result types do not match "
+ "the SP intrinsic flags");
SDLoc DL(N);
SmallVector<SDValue, 16> Operands;
Operands.reserve(N->getNumOperands() - NumFlags + 2);
- Operands.push_back(getI32Imm(ElemSize, DL));
+ Operands.push_back(getI32Imm(ElemSizeValue, DL));
Operands.push_back(getI32Imm(IdxSize, DL));
Operands.push_back(getI32Imm(RepeatFactor, DL));
for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
@@ -547,19 +560,22 @@ void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
unsigned NumTgt) {
- constexpr unsigned NumFlags = 2;
+ constexpr unsigned NumFlags = 1;
if (N->getNumOperands() < NumFlags + 3)
report_fatal_error("Malformed spdecompress intrinsic");
unsigned FlagOp = N->getNumOperands() - NumFlags;
- unsigned ElemSize =
- cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
unsigned IdxSize =
- cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
- checkSPRegTypes(N, NumFlags, "spdecompress");
+ cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
+ EVT DataRegVT = N->getValueType(0);
+ auto ElemSize = getSPDataElemSize(DataRegVT);
+ if (!ElemSize)
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
+ unsigned ElemSizeValue = *ElemSize;
// data_size = num_tgt * elem_size * repeat_factor / 32.
- unsigned RepeatFactorDenominator = NumTgt * ElemSize;
+ unsigned RepeatFactorDenominator = NumTgt * ElemSizeValue;
unsigned RepeatFactorNumerator = N->getNumValues() * 32;
if (RepeatFactorDenominator == 0 ||
RepeatFactorNumerator % RepeatFactorDenominator != 0)
@@ -570,12 +586,12 @@ void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
report_fatal_error("spdecompress operand/result types do not match "
"the SP intrinsic flags");
- auto Layout = nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSize, IdxSize,
- RepeatFactor);
+ auto Layout = nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSizeValue,
+ IdxSize, RepeatFactor);
if (!Layout)
report_fatal_error(Twine("Invalid spdecompress flags: num_src=") +
Twine(NumSrc) + ", num_tgt=" + Twine(NumTgt) +
- ", elem_size=" + Twine(ElemSize) +
+ ", elem_size=" + Twine(ElemSizeValue) +
", idx_size=" + Twine(IdxSize));
if (N->getNumValues() != Layout->DataSize ||
@@ -583,11 +599,19 @@ void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
Layout->MetadataSize + Layout->CompressedDataSize + NumFlags + 1)
report_fatal_error("spdecompress operand/result types do not match "
"the SP intrinsic flags");
+ for (unsigned I = 1; I <= Layout->MetadataSize; ++I)
+ if (N->getOperand(I).getValueType() != MVT::i32)
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
+ for (unsigned I = Layout->MetadataSize + 1; I != FlagOp; ++I)
+ if (N->getOperand(I).getValueType() != DataRegVT)
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
SDLoc DL(N);
SmallVector<SDValue, 16> Operands;
Operands.reserve(N->getNumOperands() - NumFlags + 4);
- Operands.push_back(getI32Imm(ElemSize, DL));
+ Operands.push_back(getI32Imm(ElemSizeValue, DL));
Operands.push_back(getI32Imm(IdxSize, DL));
Operands.push_back(getI32Imm(RepeatFactor, DL));
Operands.push_back(getI32Imm(NumSrc, DL));
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
index 5ed34cb547b1b3..f3a0e943605997 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
@@ -3202,95 +3202,134 @@ static SDValue lowerIntrinsicWChain(SDValue Op, SelectionDAG &DAG) {
}
}
-static bool needsSPCompressScalarization(SDNode *N) {
- bool NeedsScalarize =
- N->getValueType(0).isVector() || N->getValueType(1).isVector();
+static EVT getSPVectorPartType(EVT VT) {
+ assert(VT.isVector() && "expected a vector type");
+ EVT EltVT = VT.getVectorElementType();
+ if (EltVT == MVT::i8)
+ return MVT::v4i8;
+ if (EltVT == MVT::i16)
+ return MVT::v2i16;
+ return EltVT;
+}
+
+static unsigned getSPVectorNumParts(EVT VT) {
+ EVT PartVT = getSPVectorPartType(VT);
+ unsigned PartElts = PartVT.isVector() ? PartVT.getVectorNumElements() : 1;
+ if (VT.getVectorNumElements() % PartElts != 0)
+ report_fatal_error(
+ "SP intrinsic vector types must occupy whole 32-bit registers");
+ return VT.getVectorNumElements() / PartElts;
+}
+
+static bool needsSPVectorLowering(SDNode *N) {
+ for (EVT VT : N->values())
+ if (VT.isVector() && VT != getSPVectorPartType(VT))
+ return true;
for (const SDValue &Operand : N->ops())
- NeedsScalarize |= Operand.getValueType().isVector();
- return NeedsScalarize;
+ if (EVT VT = Operand.getValueType();
+ VT.isVector() && VT != getSPVectorPartType(VT))
+ return true;
+ return false;
+}
+
+static void splitSPVector(SDValue Vector, SmallVectorImpl<SDValue> &Parts,
+ SelectionDAG &DAG) {
+ EVT VT = Vector.getValueType();
+ EVT PartVT = getSPVectorPartType(VT);
+ if (!PartVT.isVector()) {
+ DAG.ExtractVectorElements(Vector, Parts);
+ return;
+ }
+
+ unsigned NumParts = getSPVectorNumParts(VT);
+ unsigned PartElts = PartVT.getVectorNumElements();
+ for (unsigned I = 0; I != NumParts; ++I)
+ Parts.push_back(
+ DAG.getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(Vector), PartVT, Vector,
+ DAG.getVectorIdxConstant(I * PartElts, SDLoc(Vector))));
+}
+
+static SDValue joinSPVector(EVT VT, ArrayRef<SDValue> Parts, const SDLoc &DL,
+ SelectionDAG &DAG) {
+ assert(Parts.size() == getSPVectorNumParts(VT) &&
+ "incorrect number of SP vector parts");
+ if (Parts.size() == 1 && Parts.front().getValueType() == VT)
+ return Parts.front();
+ if (Parts.front().getValueType().isVector())
+ return DAG.getNode(ISD::CONCAT_VECTORS, DL, VT, Parts);
+ return DAG.getBuildVector(VT, DL, Parts);
}
static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
SDNode *N = Op.getNode();
SDLoc DL(N);
- if (!needsSPCompressScalarization(N))
+ if (!needsSPVectorLowering(N))
return Op;
- SmallVector<SDValue, 4> Ops;
+ SmallVector<SDValue, 16> Ops;
for (const SDValue &Operand : N->ops()) {
if (Operand.getValueType().isVector())
- DAG.ExtractVectorElements(Operand, Ops);
+ splitSPVector(Operand, Ops, DAG);
else
Ops.push_back(Operand);
}
- if (N->getValueType(0).isVector() || N->getValueType(1).isVector()) {
- SmallVector<EVT, 4> ResultTys;
- for (const EVT VT : N->values()) {
- if (VT.isVector())
- ResultTys.append(VT.getVectorNumElements(), VT.getVectorElementType());
- else
- ResultTys.push_back(VT);
- }
+ SmallVector<EVT, 16> ResultTys;
+ for (const EVT VT : N->values()) {
+ if (VT.isVector())
+ ResultTys.append(getSPVectorNumParts(VT), getSPVectorPartType(VT));
+ else
+ ResultTys.push_back(VT);
+ }
- SDValue Lowered = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResultTys, Ops);
+ SDValue Lowered = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResultTys, Ops);
- SmallVector<SDValue, 2> Retvals;
- for (unsigned NewI = 0, I = 0, E = N->getNumValues(); I != E; ++I) {
- if (EVT VT = N->getValueType(I); VT.isVector()) {
- SmallVector<SDValue> Elements;
- for (auto NewE = NewI + VT.getVectorNumElements(); NewI != NewE; ++NewI)
- Elements.push_back(Lowered.getValue(NewI));
- Retvals.push_back(DAG.getBuildVector(VT, DL, Elements));
- } else {
- Retvals.push_back(Lowered.getValue(NewI));
- ++NewI;
- }
+ SmallVector<SDValue, 2> Retvals;
+ for (unsigned NewI = 0, I = 0, E = N->getNumValues(); I != E; ++I) {
+ if (EVT VT = N->getValueType(I); VT.isVector()) {
+ SmallVector<SDValue> Parts;
+ for (auto NewE = NewI + getSPVectorNumParts(VT); NewI != NewE; ++NewI)
+ Parts.push_back(Lowered.getValue(NewI));
+ Retvals.push_back(joinSPVector(VT, Parts, DL, DAG));
+ } else {
+ Retvals.push_back(Lowered.getValue(NewI));
+ ++NewI;
}
-
- return DAG.getMergeValues(Retvals, DL);
}
- return DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, N->getVTList(), Ops);
+ return DAG.getMergeValues(Retvals, DL);
}
static SDValue lowerSPDecompress(SDValue Op, SelectionDAG &DAG) {
SDNode *N = Op.getNode();
SDLoc DL(N);
EVT ResVT0 = N->getValueType(0);
- // Every valid layout with a scalar data result also has scalar mdata and
- // cdata operands, so only vector results require scalarization here.
- if (!ResVT0.isVector())
+ if (!needsSPVectorLowering(N))
return Op;
- unsigned NumElts = ResVT0.getVectorNumElements();
- EVT EltVT = ResVT0.getVectorElementType();
- SmallVector<EVT, 4> ListVTs(NumElts, EltVT);
+ unsigned NumParts = getSPVectorNumParts(ResVT0);
+ EVT PartVT = getSPVectorPartType(ResVT0);
+ SmallVector<EVT, 16> ListVTs(NumParts, PartVT);
SDVTList ResVTs = DAG.getVTList(ListVTs);
- SmallVector<SDValue, 4> Ops;
+ SmallVector<SDValue, 16> Ops;
Ops.push_back(N->getOperand(0));
for (unsigned I = 1; I < N->getNumOperands(); ++I) {
- SDValue Op = N->getOperand(I);
- EVT OpVT = Op.getValueType();
- if (OpVT.isVector()) {
- for (unsigned J = 0, E = OpVT.getVectorNumElements(); J != E; ++J)
- Ops.push_back(DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL,
- OpVT.getVectorElementType(), Op,
- DAG.getIntPtrConstant(J, DL)));
- } else {
- Ops.push_back(Op);
- }
+ SDValue Operand = N->getOperand(I);
+ if (Operand.getValueType().isVector())
+ splitSPVector(Operand, Ops, DAG);
+ else
+ Ops.push_back(Operand);
}
SDValue NewNode = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResVTs, Ops);
- SmallVector<SDValue, 4> ScalarRes;
- for (unsigned I = 0; I < NumElts; ++I)
- ScalarRes.push_back(NewNode.getValue(I));
- return DAG.getBuildVector(ResVT0, DL, ScalarRes);
+ SmallVector<SDValue, 16> Parts;
+ for (unsigned I = 0; I < NumParts; ++I)
+ Parts.push_back(NewNode.getValue(I));
+ return joinSPVector(ResVT0, Parts, DL, DAG);
}
static SDValue lowerIntrinsicWOChain(SDValue Op, SelectionDAG &DAG) {
@@ -7399,7 +7438,7 @@ static SDValue combineIntrinsicWOChain(SDNode *N,
return combineFAddWithNeg(N, DCI.DAG, IID, RoundingMode);
}
case Intrinsic::nvvm_spcompress_sp2to4:
- if (needsSPCompressScalarization(N))
+ if (needsSPVectorLowering(N))
return lowerSPCompress(SDValue(N, 0), DCI.DAG);
break;
case Intrinsic::nvvm_spdecompress_sp1to2:
@@ -7411,7 +7450,7 @@ static SDValue combineIntrinsicWOChain(SDNode *N,
case Intrinsic::nvvm_spdecompress_sp2to16:
case Intrinsic::nvvm_spdecompress_sp4to8:
case Intrinsic::nvvm_spdecompress_sp4to16: {
- if (N->getValueType(0).isVector())
+ if (needsSPVectorLowering(N))
return lowerSPDecompress(SDValue(N, 0), DCI.DAG);
break;
}
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
index 57d326a64b0846..6f80f646951fe9 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
@@ -1,5 +1,5 @@
; RUN: split-file %s %t
-; RUN: not llvm-as -disable-output < %t/bad-elem-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
+; RUN: not --crash llc < %t/bad-elem-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
@@ -7,67 +7,68 @@
; RUN: not --crash llc < %t/bad-result-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
; RUN: not --crash llc < %t/bad-data-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA-TY
-;--- bad-elem-size.ll
+;--- bad-elem-type.ll
-define { i32, i32 } @bad_spcompress_elem_size(i32 %spdesc,
- <2 x i32> %data) {
-; BAD-ELEM: immarg value 9 for arg 2 out of range set
- %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 9, i32 2)
- ret { i32, i32 } %res
+define { i32, <2 x i32> } @bad_spcompress_elem_type(i32 %spdesc,
+ <2 x i32> %data) {
+; BAD-ELEM: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 2)
+ ret { i32, <2 x i32> } %res
}
;--- bad-idx-size.ll
-define { i32, i32 } @bad_spcompress_idx_size(i32 %spdesc,
- <2 x i32> %data) {
-; BAD-IDX: immarg value 3 for arg 3 out of range set
- %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 3)
- ret { i32, i32 } %res
+define { i32, <4 x i8> } @bad_spcompress_idx_size(i32 %spdesc,
+ <8 x i8> %data) {
+; BAD-IDX: immarg value 3 for arg 2 out of range set
+ %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 3)
+ ret { i32, <4 x i8> } %res
}
;--- bad-mdata-size.ll
define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
- <8 x i32> %data) {
+ <32 x i8> %data) {
; BAD-MDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4)
- %mdata = extractvalue { i32, <4 x i32> } %res, 0
+ %res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4)
+ %mdata = extractvalue { i32, <16 x i8> } %res, 0
ret i32 %mdata
}
;--- bad-cdata-size.ll
-define <2 x i32> @bad_spcompress_compressed_data_register_count(
- i32 %spdesc, <8 x i32> %data) {
+define <8 x i8> @bad_spcompress_compressed_data_register_count(
+ i32 %spdesc, <32 x i8> %data) {
; BAD-CDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { <2 x i32>, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v2i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4)
- %cdata = extractvalue { <2 x i32>, <2 x i32> } %res, 1
- ret <2 x i32> %cdata
+ %res = call { <2 x i32>, <8 x i8> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4)
+ %cdata = extractvalue { <2 x i32>, <8 x i8> } %res, 1
+ ret <8 x i8> %cdata
}
;--- bad-data-size.ll
-define <4 x i32> @bad_spcompress_data_register_count(i32 %spdesc,
- <6 x i32> %data) {
+define <16 x i8> @bad_spcompress_data_register_count(i32 %spdesc,
+ <24 x i8> %data) {
; BAD-DATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v6i32(<6 x i32> %data, i32 %spdesc, i32 8, i32 4)
- %cdata = extractvalue { <2 x i32>, <4 x i32> } %res, 1
- ret <4 x i32> %cdata
+ %res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i8.v24i8(<24 x i8> %data, i32 %spdesc, i32 4)
+ %cdata = extractvalue { <2 x i32>, <16 x i8> } %res, 1
+ ret <16 x i8> %cdata
}
;--- bad-result-type.ll
-define i64 @bad_spcompress_result_type(i32 %spdesc, <2 x i32> %data) {
-; BAD-RESULT-TY: LLVM ERROR: spcompress expects scalarized i32 results
- %res = call { i64, i32 } @llvm.nvvm.spcompress.sp2to4.i64.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2)
- %mdata = extractvalue { i64, i32 } %res, 0
+define i64 @bad_spcompress_result_type(i32 %spdesc, <8 x i8> %data) {
+; BAD-RESULT-TY: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { i64, <4 x i8> } @llvm.nvvm.spcompress.sp2to4.i64.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2)
+ %mdata = extractvalue { i64, <4 x i8> } %res, 0
ret i64 %mdata
}
;--- bad-data-type.ll
-define { i32, i32 } @bad_spcompress_data_type(i32 %spdesc, i64 %data) {
-; BAD-DATA-TY: LLVM ERROR: spcompress expects scalarized i32 operands
- %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.i64(i64 %data, i32 %spdesc, i32 8, i32 2)
- ret { i32, i32 } %res
+define { i32, <2 x i16> } @bad_spcompress_data_type(i32 %spdesc,
+ <8 x i8> %data) {
+; BAD-DATA-TY: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v8i8(<8 x i8> %data, i32 %spdesc, i32 2)
+ ret { i32, <2 x i16> } %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
index d12a3d03a5fb10..ff19ab469fa4ac 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
@@ -3,9 +3,9 @@
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
-define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32> %data) {
-; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <8 x i8> %data) {
+; FORMAT-LABEL: define { i32, <4 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -17,13 +17,13 @@ define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2)
- ret {i32, i32} %res
+ %res = call {i32, <4 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2)
+ ret {i32, <4 x i8>} %res
}
-define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32> %data) {
-; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <4 x i16> %data) {
+; FORMAT-LABEL: define { i32, <2 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -35,13 +35,13 @@ define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <2 x i3
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 2)
- ret {i32, i32} %res
+ %res = call {i32, <2 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, i32 2)
+ ret {i32, <2 x i16>} %res
}
-define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32> %data) {
-; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <8 x i8> %data) {
+; FORMAT-LABEL: define { i32, <4 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -53,13 +53,13 @@ define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 4)
- ret {i32, i32} %res
+ %res = call {i32, <4 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 4)
+ ret {i32, <4 x i8>} %res
}
-define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32> %data) {
-; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <4 x i16> %data) {
+; FORMAT-LABEL: define { i32, <2 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -71,13 +71,13 @@ define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <2 x i3
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 4)
- ret {i32, i32} %res
+ %res = call {i32, <2 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, i32 4)
+ ret {i32, <2 x i16>} %res
}
-define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <4 x i32> %data) {
-; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <16 x i8> %data) {
+; FORMAT-LABEL: define { i32, <8 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -89,13 +89,13 @@ define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <4
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 2)
- ret {i32, <2 x i32>} %res
+ %res = call {i32, <8 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, i32 2)
+ ret {i32, <8 x i8>} %res
}
-define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <4 x i32> %data) {
-; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <8 x i16> %data) {
+; FORMAT-LABEL: define { i32, <4 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -107,13 +107,13 @@ define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 2)
- ret {i32, <2 x i32>} %res
+ %res = call {i32, <4 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, i32 2)
+ ret {i32, <4 x i16>} %res
}
-define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <4 x i32> %data) {
-; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <16 x i8> %data) {
+; FORMAT-LABEL: define { i32, <8 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -125,13 +125,13 @@ define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <4
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 4)
- ret {i32, <2 x i32>} %res
+ %res = call {i32, <8 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, i32 4)
+ ret {i32, <8 x i8>} %res
}
-define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <4 x i32> %data) {
-; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <8 x i16> %data) {
+; FORMAT-LABEL: define { i32, <4 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -143,13 +143,13 @@ define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 4)
- ret {i32, <2 x i32>} %res
+ %res = call {i32, <4 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, i32 4)
+ ret {i32, <4 x i16>} %res
}
-define {i32, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <8 x i32> %data) {
-; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define {i32, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <32 x i8> %data) {
+; FORMAT-LABEL: define { i32, <16 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -162,13 +162,13 @@ define {i32, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <8
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 2)
- ret {i32, <4 x i32>} %res
+ %res = call {i32, <16 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 2)
+ ret {i32, <16 x i8>} %res
}
-define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <8 x i32> %data) {
-; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <16 x i16> %data) {
+; FORMAT-LABEL: define { i32, <8 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -181,13 +181,13 @@ define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 2)
- ret {i32, <4 x i32>} %res
+ %res = call {i32, <8 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, i32 2)
+ ret {i32, <8 x i16>} %res
}
-define {<2 x i32>, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spdesc, <8 x i32> %data) {
-; FORMAT-LABEL: define { <2 x i32>, <4 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define {<2 x i32>, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spdesc, <32 x i8> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <16 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<16>;
@@ -200,13 +200,13 @@ define {<2 x i32>, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r12, %r13, %r14, %r15};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r10, %r11};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4)
- ret {<2 x i32>, <4 x i32>} %res
+ %res = call {<2 x i32>, <16 x i8>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4)
+ ret {<2 x i32>, <16 x i8>} %res
}
-define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <8 x i32> %data) {
-; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <16 x i16> %data) {
+; FORMAT-LABEL: define { i32, <8 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -219,13 +219,13 @@ define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 4)
- ret {i32, <4 x i32>} %res
+ %res = call {i32, <8 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, i32 4)
+ ret {i32, <8 x i16>} %res
}
-define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spdesc, <16 x i32> %data) {
-; FORMAT-LABEL: define { <2 x i32>, <8 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define {<2 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spdesc, <64 x i8> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <32 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i8> } @llvm.nvvm.spcompress.sp2to4.v2i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<28>;
@@ -241,13 +241,13 @@ define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 2)
- ret {<2 x i32>, <8 x i32>} %res
+ %res = call {<2 x i32>, <32 x i8>} @llvm.nvvm.spcompress.sp2to4.v2i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, i32 2)
+ ret {<2 x i32>, <32 x i8>} %res
}
-define {i32, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <16 x i32> %data) {
-; FORMAT-LABEL: define { i32, <8 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call { i32, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define {i32, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <32 x i16> %data) {
+; FORMAT-LABEL: define { i32, <16 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call { i32, <16 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -263,13 +263,13 @@ define {i32, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r18;
; CHECK-NEXT: ret;
- %res = call {i32, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 2)
- ret {i32, <8 x i32>} %res
+ %res = call {i32, <16 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, i32 2)
+ ret {i32, <16 x i16>} %res
}
-define {<4 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spdesc, <16 x i32> %data) {
-; FORMAT-LABEL: define { <4 x i32>, <8 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call { <4 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define {<4 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spdesc, <64 x i8> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <32 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i8> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<30>;
@@ -285,13 +285,13 @@ define {<4 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r22, %r23, %r24, %r25};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r18, %r19, %r20, %r21};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 4)
- ret {<4 x i32>, <8 x i32>} %res
+ %res = call {<4 x i32>, <32 x i8>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, i32 4)
+ ret {<4 x i32>, <32 x i8>} %res
}
-define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spdesc, <16 x i32> %data) {
-; FORMAT-LABEL: define { <2 x i32>, <8 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define {<2 x i32>, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spdesc, <32 x i16> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <16 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i16> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<28>;
@@ -307,13 +307,13 @@ define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spd
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 4)
- ret {<2 x i32>, <8 x i32>} %res
+ %res = call {<2 x i32>, <16 x i16>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, i32 4)
+ ret {<2 x i32>, <16 x i16>} %res
}
-define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spdesc, <32 x i32> %data) {
-; FORMAT-LABEL: define { <4 x i32>, <16 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define {<4 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spdesc, <128 x i8> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <64 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i8> } @llvm.nvvm.spcompress.sp2to4.v4i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<54>;
@@ -335,13 +335,13 @@ define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %sp
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 2)
- ret {<4 x i32>, <16 x i32>} %res
+ %res = call {<4 x i32>, <64 x i8>} @llvm.nvvm.spcompress.sp2to4.v4i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, i32 2)
+ ret {<4 x i32>, <64 x i8>} %res
}
-define {<2 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %spdesc, <32 x i32> %data) {
-; FORMAT-LABEL: define { <2 x i32>, <16 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define {<2 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %spdesc, <64 x i16> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <32 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i16> } @llvm.nvvm.spcompress.sp2to4.v2i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<52>;
@@ -363,13 +363,13 @@ define {<2 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r36, %r37, %r38, %r39};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r34, %r35};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 2)
- ret {<2 x i32>, <16 x i32>} %res
+ %res = call {<2 x i32>, <32 x i16>} @llvm.nvvm.spcompress.sp2to4.v2i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, i32 2)
+ ret {<2 x i32>, <32 x i16>} %res
}
-define {<8 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spdesc, <32 x i32> %data) {
-; FORMAT-LABEL: define { <8 x i32>, <16 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call { <8 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define {<8 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spdesc, <128 x i8> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <64 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i8> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<58>;
@@ -392,13 +392,13 @@ define {<8 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %sp
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 4)
- ret {<8 x i32>, <16 x i32>} %res
+ %res = call {<8 x i32>, <64 x i8>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, i32 4)
+ ret {<8 x i32>, <64 x i8>} %res
}
-define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %spdesc, <32 x i32> %data) {
-; FORMAT-LABEL: define { <4 x i32>, <16 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define {<4 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %spdesc, <64 x i16> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <32 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i16> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<54>;
@@ -420,13 +420,13 @@ define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 4)
- ret {<4 x i32>, <16 x i32>} %res
+ %res = call {<4 x i32>, <32 x i16>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, i32 4)
+ ret {<4 x i32>, <32 x i16>} %res
}
-define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %spdesc, <64 x i32> %data) {
-; FORMAT-LABEL: define { <8 x i32>, <32 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define {<8 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %spdesc, <256 x i8> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <128 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i8> } @llvm.nvvm.spcompress.sp2to4.v8i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<106>;
@@ -461,13 +461,13 @@ define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %sp
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 2)
- ret {<8 x i32>, <32 x i32>} %res
+ %res = call {<8 x i32>, <128 x i8>} @llvm.nvvm.spcompress.sp2to4.v8i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, i32 2)
+ ret {<8 x i32>, <128 x i8>} %res
}
-define {<4 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %spdesc, <64 x i32> %data) {
-; FORMAT-LABEL: define { <4 x i32>, <32 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define {<4 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %spdesc, <128 x i16> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <64 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i16> } @llvm.nvvm.spcompress.sp2to4.v4i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<102>;
@@ -501,13 +501,13 @@ define {<4 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 2)
- ret {<4 x i32>, <32 x i32>} %res
+ %res = call {<4 x i32>, <64 x i16>} @llvm.nvvm.spcompress.sp2to4.v4i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, i32 2)
+ ret {<4 x i32>, <64 x i16>} %res
}
-define {<16 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %spdesc, <64 x i32> %data) {
-; FORMAT-LABEL: define { <16 x i32>, <32 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call { <16 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define {<16 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %spdesc, <256 x i8> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <128 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i8> } @llvm.nvvm.spcompress.sp2to4.v16i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<114>;
@@ -544,13 +544,13 @@ define {<16 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 4)
- ret {<16 x i32>, <32 x i32>} %res
+ %res = call {<16 x i32>, <128 x i8>} @llvm.nvvm.spcompress.sp2to4.v16i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, i32 4)
+ ret {<16 x i32>, <128 x i8>} %res
}
-define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %spdesc, <64 x i32> %data) {
-; FORMAT-LABEL: define { <8 x i32>, <32 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define {<8 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %spdesc, <128 x i16> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <64 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i16> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<106>;
@@ -585,13 +585,13 @@ define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 4)
- ret {<8 x i32>, <32 x i32>} %res
+ %res = call {<8 x i32>, <64 x i16>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, i32 4)
+ ret {<8 x i32>, <64 x i16>} %res
}
-define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %spdesc, <128 x i32> %data) {
-; FORMAT-LABEL: define { <16 x i32>, <64 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define {<16 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %spdesc, <512 x i8> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <256 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call { <16 x i32>, <256 x i8> } @llvm.nvvm.spcompress.sp2to4.v16i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<210>;
@@ -652,13 +652,13 @@ define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 2)
- ret {<16 x i32>, <64 x i32>} %res
+ %res = call {<16 x i32>, <256 x i8>} @llvm.nvvm.spcompress.sp2to4.v16i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, i32 2)
+ ret {<16 x i32>, <256 x i8>} %res
}
-define {<8 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %spdesc, <128 x i32> %data) {
-; FORMAT-LABEL: define { <8 x i32>, <64 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define {<8 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %spdesc, <256 x i16> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <128 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i16> } @llvm.nvvm.spcompress.sp2to4.v8i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<202>;
@@ -717,13 +717,13 @@ define {<8 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 2)
- ret {<8 x i32>, <64 x i32>} %res
+ %res = call {<8 x i32>, <128 x i16>} @llvm.nvvm.spcompress.sp2to4.v8i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, i32 2)
+ ret {<8 x i32>, <128 x i16>} %res
}
-define {<32 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %spdesc, <128 x i32> %data) {
-; FORMAT-LABEL: define { <32 x i32>, <64 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call { <32 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define {<32 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %spdesc, <512 x i8> %data) {
+; FORMAT-LABEL: define { <32 x i32>, <256 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call { <32 x i32>, <256 x i8> } @llvm.nvvm.spcompress.sp2to4.v32i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<226>;
@@ -788,13 +788,13 @@ define {<32 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<32 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 4)
- ret {<32 x i32>, <64 x i32>} %res
+ %res = call {<32 x i32>, <256 x i8>} @llvm.nvvm.spcompress.sp2to4.v32i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, i32 4)
+ ret {<32 x i32>, <256 x i8>} %res
}
-define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %spdesc, <128 x i32> %data) {
-; FORMAT-LABEL: define { <16 x i32>, <64 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define {<16 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %spdesc, <256 x i16> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <128 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i16> } @llvm.nvvm.spcompress.sp2to4.v16i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<210>;
@@ -855,6 +855,6 @@ define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 4)
- ret {<16 x i32>, <64 x i32>} %res
+ %res = call {<16 x i32>, <128 x i16>} @llvm.nvvm.spcompress.sp2to4.v16i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, i32 4)
+ ret {<16 x i32>, <128 x i16>} %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
index 18e9e5cfa460f1..07f494ac7b58f5 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
@@ -1,5 +1,5 @@
; RUN: split-file %s %t
-; RUN: not llvm-as -disable-output < %t/bad-elem-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
+; RUN: not --crash llc < %t/bad-elem-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
; RUN: not --crash llc < %t/bad-idx-size-for-target.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-IDX-TARGET
; RUN: not --crash llc < %t/bad-input-span.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-INPUT-SPAN
@@ -9,79 +9,83 @@
; RUN: not --crash llc < %t/bad-result-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
; RUN: not --crash llc < %t/bad-mdata-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA-TY
-;--- bad-elem-size.ll
+;--- bad-elem-type.ll
-define i32 @bad_spdecompress_elem_size(i32 %metadata, i32 %compdata) {
-; BAD-ELEM: immarg value 9 for arg 2 out of range set
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 9, i32 2)
- ret i32 %res
+define <2 x i32> @bad_spdecompress_elem_type(i32 %metadata,
+ <2 x i32> %compdata) {
+; BAD-ELEM: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 2)
+ ret <2 x i32> %res
}
;--- bad-idx-size.ll
-define i32 @bad_spdecompress_idx_size(i32 %metadata, i32 %compdata) {
-; BAD-IDX: immarg value 3 for arg 3 out of range set
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 3)
- ret i32 %res
+define <4 x i8> @bad_spdecompress_idx_size(i32 %metadata,
+ <4 x i8> %compdata) {
+; BAD-IDX: immarg value 3 for arg 2 out of range set
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 3)
+ ret <4 x i8> %res
}
;--- bad-idx-size-for-target.ll
-define <8 x i32> @bad_spdecompress_idx_size_for_target(
- i32 %metadata, i32 %compdata) {
+define <32 x i8> @bad_spdecompress_idx_size_for_target(
+ i32 %metadata, <4 x i8> %compdata) {
; BAD-IDX-TARGET: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=8, elem_size=8, idx_size=2
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to8.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <32 x i8> %res
}
;--- bad-input-span.ll
-define <4 x i32> @bad_spdecompress_input_span(<2 x i32> %metadata,
- <2 x i32> %compdata) {
+define <8 x i16> @bad_spdecompress_input_span(<2 x i32> %metadata,
+ <4 x i16> %compdata) {
; BAD-INPUT-SPAN: LLVM ERROR: Invalid spdecompress flags: num_src=4, num_tgt=8, elem_size=16, idx_size=4
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 16, i32 4)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp4to8.v8i16.v2i32.v4i16(<2 x i32> %metadata, <4 x i16> %compdata, i32 4)
+ ret <8 x i16> %res
}
;--- bad-mdata-size.ll
-define <4 x i32> @bad_spdecompress_metadata_register_count(
- <2 x i32> %metadata, <2 x i32> %compdata) {
+define <16 x i8> @bad_spdecompress_metadata_register_count(
+ <2 x i32> %metadata, <8 x i8> %compdata) {
; BAD-MDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.v2i32.v8i8(<2 x i32> %metadata, <8 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
;--- bad-cdata-size.ll
-define <4 x i32> @bad_spdecompress_compressed_data_register_count(
- i32 %metadata, i32 %compdata) {
+define <8 x i16> @bad_spdecompress_compressed_data_register_count(
+ i32 %metadata, <2 x i16> %compdata) {
; BAD-CDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <8 x i16> %res
}
;--- bad-data-size.ll
-define <3 x i32> @bad_spdecompress_data_register_count(
- i32 %metadata, <2 x i32> %compdata) {
+define <12 x i8> @bad_spdecompress_data_register_count(
+ i32 %metadata, <8 x i8> %compdata) {
; BAD-DATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <3 x i32> @llvm.nvvm.spdecompress.sp2to4.v3i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <3 x i32> %res
+ %res = call <12 x i8> @llvm.nvvm.spdecompress.sp2to4.v12i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <12 x i8> %res
}
;--- bad-result-type.ll
-define i64 @bad_spdecompress_result_type(i32 %metadata, i32 %compdata) {
-; BAD-RESULT-TY: LLVM ERROR: spdecompress expects scalarized i32 results
- %res = call i64 @llvm.nvvm.spdecompress.sp2to4.i64.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret i64 %res
+define <4 x i16> @bad_spdecompress_result_type(i32 %metadata,
+ <4 x i8> %compdata) {
+; BAD-RESULT-TY: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <4 x i16> %res
}
;--- bad-mdata-type.ll
-define i32 @bad_spdecompress_metadata_type(i64 %metadata, i32 %compdata) {
-; BAD-MDATA-TY: LLVM ERROR: spdecompress expects scalarized i32 operands
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i64.i32(i64 %metadata, i32 %compdata, i32 8, i32 2)
- ret i32 %res
+define <4 x i8> @bad_spdecompress_metadata_type(i64 %metadata,
+ <4 x i8> %compdata) {
+; BAD-MDATA-TY: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i64.v4i8(i64 %metadata, <4 x i8> %compdata, i32 2)
+ ret <4 x i8> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
index 52abb3c66a937d..6359c22a8dbf72 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
@@ -4,9 +4,9 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4
-define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -17,13 +17,13 @@ define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
-define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to16.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -35,13 +35,13 @@ define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to16.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
-define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to16.v64i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<19>;
@@ -55,13 +55,13 @@ define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to16.v64i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
-define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to16.v128i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<36>;
@@ -79,13 +79,13 @@ define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to16.v128i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
-define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <256 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to16.v256i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<71>;
@@ -111,13 +111,13 @@ define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to16.v256i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <256 x i8> %res
}
-define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <512 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp1to16.v512i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<141>;
@@ -160,14 +160,14 @@ define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <128 x i32> %res
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.sp1to16.v512i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <512 x i8> %res
}
; elem_size=16, idx_size=4
-define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to16.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -179,13 +179,13 @@ define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to16.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <16 x i16> %res
}
-define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <32 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to16.v32i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<19>;
@@ -199,13 +199,13 @@ define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to16.v32i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <32 x i16> %res
}
-define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <64 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to16.v64i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<36>;
@@ -223,13 +223,13 @@ define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to16.v64i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ ret <64 x i16> %res
}
-define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <128 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to16.v128i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<70>;
@@ -255,13 +255,13 @@ define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to16.v128i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ ret <128 x i16> %res
}
-define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <256 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to16.v256i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<139>;
@@ -304,6 +304,6 @@ define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
- ret <128 x i32> %res
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to16.v256i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
index 8a7260f878643d..6396c02ab58fb6 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
@@ -3,9 +3,9 @@
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
-define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to2.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -16,13 +16,13 @@ define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x2 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret i32 %res
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to2.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <4 x i8> %res
}
-define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to2.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -33,13 +33,13 @@ define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to2.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <8 x i8> %res
}
-define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to2.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -50,13 +50,13 @@ define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to2.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2)
+ ret <16 x i8> %res
}
-define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to2.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -68,13 +68,13 @@ define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to2.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2)
+ ret <32 x i8> %res
}
-define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to2.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -89,13 +89,13 @@ define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to2.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2)
+ ret <64 x i8> %res
}
-define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to2.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -116,13 +116,13 @@ define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to2.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2)
+ ret <128 x i8> %res
}
-define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to2.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -133,13 +133,13 @@ define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x2 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret i32 %res
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to2.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <4 x i8> %res
}
-define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to2.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -150,13 +150,13 @@ define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to2.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <8 x i8> %res
}
-define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to2.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -167,13 +167,13 @@ define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to2.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
-define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to2.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -185,13 +185,13 @@ define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to2.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
-define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to2.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -206,13 +206,13 @@ define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to2.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
-define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to2.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -234,13 +234,13 @@ define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to2.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
-define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.sp1to2.v2i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -251,13 +251,13 @@ define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
- ret i32 %res
+ %res = call <2 x i16> @llvm.nvvm.spdecompress.sp1to2.v2i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ ret <2 x i16> %res
}
-define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to2.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -268,13 +268,13 @@ define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
- ret <2 x i32> %res
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to2.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ ret <4 x i16> %res
}
-define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to2.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -285,13 +285,13 @@ define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to2.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2)
+ ret <8 x i16> %res
}
-define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to2.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -303,13 +303,13 @@ define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to2.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2)
+ ret <16 x i16> %res
}
-define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to2.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<26>;
@@ -324,13 +324,13 @@ define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to2.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2)
+ ret <32 x i16> %res
}
-define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to2.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<51>;
@@ -351,13 +351,13 @@ define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to2.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2)
+ ret <64 x i16> %res
}
-define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to2.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<101>;
@@ -390,13 +390,13 @@ define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to2.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2)
+ ret <128 x i16> %res
}
-define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.sp1to2.v2i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -407,13 +407,13 @@ define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret i32 %res
+ %res = call <2 x i16> @llvm.nvvm.spdecompress.sp1to2.v2i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <2 x i16> %res
}
-define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to2.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -424,13 +424,13 @@ define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <2 x i32> %res
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to2.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <4 x i16> %res
}
-define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to2.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -441,13 +441,13 @@ define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to2.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ ret <8 x i16> %res
}
-define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to2.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -459,13 +459,13 @@ define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to2.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ ret <16 x i16> %res
}
-define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to2.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -480,13 +480,13 @@ define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to2.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ ret <32 x i16> %res
}
-define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to2.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -507,13 +507,13 @@ define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to2.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ ret <64 x i16> %res
}
-define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to2.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -547,6 +547,6 @@ define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to2.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4)
+ ret <128 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
index b30695cba90351..f32a0dffe68bf4 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
@@ -4,9 +4,9 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=2
-define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -17,13 +17,13 @@ define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret i32 %res
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <4 x i8> %res
}
-define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -34,13 +34,13 @@ define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <8 x i8> %res
}
-define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to4.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -51,13 +51,13 @@ define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to4.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <16 x i8> %res
}
-define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to4.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -69,13 +69,13 @@ define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to4.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2)
+ ret <32 x i8> %res
}
-define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to4.v64i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -89,13 +89,13 @@ define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to4.v64i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2)
+ ret <64 x i8> %res
}
-define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to4.v128i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -114,13 +114,13 @@ define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to4.v128i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2)
+ ret <128 x i8> %res
}
-define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to4.v256i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -149,14 +149,14 @@ define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to4.v256i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2)
+ ret <256 x i8> %res
}
; elem_size=8, idx_size=4
-define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -167,13 +167,13 @@ define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret i32 %res
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <4 x i8> %res
}
-define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -184,13 +184,13 @@ define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <8 x i8> %res
}
-define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to4.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -201,13 +201,13 @@ define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to4.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
-define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to4.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -219,13 +219,13 @@ define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to4.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
-define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to4.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -239,13 +239,13 @@ define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to4.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
-define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to4.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -264,13 +264,13 @@ define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to4.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
-define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to4.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -300,14 +300,14 @@ define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to4.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ ret <256 x i8> %res
}
; elem_size=16, idx_size=2
-define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -318,13 +318,13 @@ define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
- ret <2 x i32> %res
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ ret <4 x i16> %res
}
-define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -335,13 +335,13 @@ define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ ret <8 x i16> %res
}
-define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to4.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -353,13 +353,13 @@ define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to4.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2)
+ ret <16 x i16> %res
}
-define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to4.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -373,13 +373,13 @@ define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to4.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2)
+ ret <32 x i16> %res
}
-define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to4.v64i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<42>;
@@ -398,13 +398,13 @@ define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to4.v64i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2)
+ ret <64 x i16> %res
}
-define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to4.v128i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<83>;
@@ -433,13 +433,13 @@ define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to4.v128i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2)
+ ret <128 x i16> %res
}
-define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i16> %compdata) {
+; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to4.v256i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<165>;
@@ -488,14 +488,14 @@ define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2)
- ret <128 x i32> %res
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to4.v256i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2)
+ ret <256 x i16> %res
}
; elem_size=16, idx_size=4
-define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -506,13 +506,13 @@ define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <2 x i32> %res
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <4 x i16> %res
}
-define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -523,13 +523,13 @@ define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <8 x i16> %res
}
-define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to4.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -541,13 +541,13 @@ define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to4.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ ret <16 x i16> %res
}
-define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to4.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -561,13 +561,13 @@ define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to4.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ ret <32 x i16> %res
}
-define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to4.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -586,13 +586,13 @@ define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to4.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ ret <64 x i16> %res
}
-define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to4.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -621,13 +621,13 @@ define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to4.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ ret <128 x i16> %res
}
-define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i16> %compdata) {
+; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to4.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<169>;
@@ -677,6 +677,6 @@ define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4)
- ret <128 x i32> %res
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to4.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4)
+ ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
index f98b15bc84c92a..d55af2e3fcae8e 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
@@ -4,9 +4,9 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4
-define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -17,13 +17,13 @@ define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <8 x i8> %res
}
-define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to8.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -34,13 +34,13 @@ define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to8.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
-define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to8.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -52,13 +52,13 @@ define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to8.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
-define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to8.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -72,13 +72,13 @@ define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to8.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
-define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to8.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<39>;
@@ -96,13 +96,13 @@ define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to8.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
-define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <256 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to8.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<77>;
@@ -129,13 +129,13 @@ define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to8.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <256 x i8> %res
}
-define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <512 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp1to8.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<153>;
@@ -181,14 +181,14 @@ define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
- ret <128 x i32> %res
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.sp1to8.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ ret <512 x i8> %res
}
; elem_size=16, idx_size=4
-define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to8.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -199,13 +199,13 @@ define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to8.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <8 x i16> %res
}
-define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <16 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to8.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -217,13 +217,13 @@ define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to8.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <16 x i16> %res
}
-define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <32 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to8.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -237,13 +237,13 @@ define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to8.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ ret <32 x i16> %res
}
-define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <64 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to8.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<38>;
@@ -261,13 +261,13 @@ define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to8.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ ret <64 x i16> %res
}
-define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <128 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to8.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<75>;
@@ -294,13 +294,13 @@ define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to8.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ ret <128 x i16> %res
}
-define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <256 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to8.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<149>;
@@ -345,6 +345,6 @@ define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
- ret <128 x i32> %res
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to8.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
index 7b3e1d8261aa89..120fe77e1520a1 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
@@ -4,9 +4,9 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
-define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -17,14 +17,14 @@ define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
-define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to16.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -36,14 +36,14 @@ define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to16.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
-define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to16.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -57,14 +57,14 @@ define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to16.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
-define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to16.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<39>;
@@ -82,14 +82,14 @@ define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to16.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
-define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <256 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to16.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<77>;
@@ -116,14 +116,14 @@ define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to16.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
-define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <512 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp2to16.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<153>;
@@ -169,14 +169,14 @@ define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
- ret <128 x i32> %res
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.sp2to16.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ ret <512 x i8> %res
}
; elem_size=16, idx_size=4, rpf=1
-define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <16 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to16.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -188,14 +188,14 @@ define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to16.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <16 x i16> %res
}
; elem_size=16, idx_size=4, rpf=2
-define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <32 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to16.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -209,14 +209,14 @@ define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to16.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ ret <32 x i16> %res
}
; elem_size=16, idx_size=4, rpf=4
-define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <64 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to16.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<38>;
@@ -234,14 +234,14 @@ define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to16.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ ret <64 x i16> %res
}
; elem_size=16, idx_size=4, rpf=8
-define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <128 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to16.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<75>;
@@ -268,14 +268,14 @@ define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to16.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ ret <128 x i16> %res
}
; elem_size=16, idx_size=4, rpf=16
-define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <256 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to16.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<149>;
@@ -320,6 +320,6 @@ define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
- ret <128 x i32> %res
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to16.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
index be01b26892052e..88ad05acd0945e 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
@@ -4,9 +4,9 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=2, rpf=1
-define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -17,14 +17,14 @@ define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret i32 %res
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <4 x i8> %res
}
; elem_size=8, idx_size=2, rpf=2
-define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -35,14 +35,14 @@ define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ ret <8 x i8> %res
}
; elem_size=8, idx_size=2, rpf=4
-define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -53,14 +53,14 @@ define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2)
+ ret <16 x i8> %res
}
; elem_size=8, idx_size=2, rpf=8
-define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to4.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -72,14 +72,14 @@ define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to4.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2)
+ ret <32 x i8> %res
}
; elem_size=8, idx_size=2, rpf=16
-define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to4.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -94,14 +94,14 @@ define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to4.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2)
+ ret <64 x i8> %res
}
; elem_size=8, idx_size=2, rpf=32
-define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to4.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -122,14 +122,14 @@ define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to4.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2)
+ ret <128 x i8> %res
}
; elem_size=8, idx_size=2, rpf=64
-define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2)
+define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %metadata, <128 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to4.v256i8.v8i32.v128i8(<8 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -163,14 +163,14 @@ define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 2)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to4.v256i8.v8i32.v128i8(<8 x i32> %metadata, <128 x i8> %compdata, i32 2)
+ ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=1
-define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<4>;
@@ -181,14 +181,14 @@ define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32
; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x1 {%r3}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret i32 %res
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <4 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
-define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -199,14 +199,14 @@ define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <8 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
-define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -217,14 +217,14 @@ define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
-define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to4.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -236,14 +236,14 @@ define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %me
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to4.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
-define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to4.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -258,14 +258,14 @@ define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to4.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
-define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to4.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -287,14 +287,14 @@ define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to4.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=64
-define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to4.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<113>;
@@ -330,14 +330,14 @@ define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to4.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4)
+ ret <256 x i8> %res
}
; elem_size=16, idx_size=2, rpf=1
-define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -348,14 +348,14 @@ define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2)
- ret <2 x i32> %res
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ ret <4 x i16> %res
}
; elem_size=16, idx_size=2, rpf=2
-define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -366,14 +366,14 @@ define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2)
+ ret <8 x i16> %res
}
; elem_size=16, idx_size=2, rpf=4
-define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to4.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -385,14 +385,14 @@ define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to4.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2)
+ ret <16 x i16> %res
}
; elem_size=16, idx_size=2, rpf=8
-define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to4.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<26>;
@@ -407,14 +407,14 @@ define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to4.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2)
+ ret <32 x i16> %res
}
; elem_size=16, idx_size=2, rpf=16
-define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to4.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<51>;
@@ -435,14 +435,14 @@ define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to4.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2)
+ ret <64 x i16> %res
}
; elem_size=16, idx_size=2, rpf=32
-define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32> %metadata, <64 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to4.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<101>;
@@ -475,14 +475,14 @@ define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to4.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2)
+ ret <128 x i16> %res
}
; elem_size=16, idx_size=2, rpf=64
-define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32> %metadata, <64 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2)
+define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32> %metadata, <128 x i16> %compdata) {
+; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to4.v256i16.v8i32.v128i16(<8 x i32> %metadata, <128 x i16> %compdata, /* idx_size=.b2 */ i32 2)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<201>;
@@ -540,14 +540,14 @@ define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r77, %r78, %r79, %r80};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r73, %r74, %r75, %r76};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 2)
- ret <128 x i32> %res
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to4.v256i16.v8i32.v128i16(<8 x i32> %metadata, <128 x i16> %compdata, i32 2)
+ ret <256 x i16> %res
}
; elem_size=16, idx_size=4, rpf=1
-define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -558,14 +558,14 @@ define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <2 x i32> %res
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <4 x i16> %res
}
; elem_size=16, idx_size=4, rpf=2
-define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -576,14 +576,14 @@ define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ ret <8 x i16> %res
}
; elem_size=16, idx_size=4, rpf=4
-define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to4.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -595,14 +595,14 @@ define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to4.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ ret <16 x i16> %res
}
; elem_size=16, idx_size=4, rpf=8
-define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to4.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -617,14 +617,14 @@ define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to4.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ ret <32 x i16> %res
}
; elem_size=16, idx_size=4, rpf=16
-define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to4.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -645,14 +645,14 @@ define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to4.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ ret <64 x i16> %res
}
; elem_size=16, idx_size=4, rpf=32
-define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to4.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -686,14 +686,14 @@ define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to4.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4)
+ ret <128 x i16> %res
}
; elem_size=16, idx_size=4, rpf=64
-define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32> %metadata, <64 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i16> %compdata) {
+; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to4.v256i16.v16i32.v128i16(<16 x i32> %metadata, <128 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<209>;
@@ -753,6 +753,6 @@ define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r85, %r86, %r87, %r88};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r81, %r82, %r83, %r84};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 4)
- ret <128 x i32> %res
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to4.v256i16.v16i32.v128i16(<16 x i32> %metadata, <128 x i16> %compdata, i32 4)
+ ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
index 206ce5a560697d..f27d04cadec7f1 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
@@ -4,9 +4,9 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
-define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -17,14 +17,14 @@ define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <8 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
-define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to8.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -35,14 +35,14 @@ define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to8.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
-define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to8.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -54,14 +54,14 @@ define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to8.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
-define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to8.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -75,14 +75,14 @@ define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to8.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
-define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to8.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -101,14 +101,14 @@ define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to8.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
-define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <256 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to8.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -138,14 +138,14 @@ define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to8.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=64
-define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <512 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i8> %compdata) {
+; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp2to8.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<177>;
@@ -197,14 +197,14 @@ define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4)
- ret <128 x i32> %res
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.sp2to8.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4)
+ ret <512 x i8> %res
}
; elem_size=16, idx_size=4, rpf=1
-define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <8 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to8.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -215,14 +215,14 @@ define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4)
- ret <4 x i32> %res
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to8.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ ret <8 x i16> %res
}
; elem_size=16, idx_size=4, rpf=2
-define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <16 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
+; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to8.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -234,14 +234,14 @@ define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4)
- ret <8 x i32> %res
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to8.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ ret <16 x i16> %res
}
; elem_size=16, idx_size=4, rpf=4
-define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <32 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
+; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to8.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -255,14 +255,14 @@ define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4)
- ret <16 x i32> %res
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to8.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ ret <32 x i16> %res
}
; elem_size=16, idx_size=4, rpf=8
-define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <64 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
+; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to8.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -281,14 +281,14 @@ define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4)
- ret <32 x i32> %res
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to8.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ ret <64 x i16> %res
}
; elem_size=16, idx_size=4, rpf=16
-define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <128 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
+; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to8.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -317,14 +317,14 @@ define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4)
- ret <64 x i32> %res
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to8.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ ret <128 x i16> %res
}
; elem_size=16, idx_size=4, rpf=32
-define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4)
+define <256 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i16> %compdata) {
+; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to8.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<169>;
@@ -374,6 +374,6 @@ define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4)
- ret <128 x i32> %res
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to8.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4)
+ ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
index 3b8d80c20d8643..dae7437d62cafa 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
@@ -4,9 +4,9 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
-define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp4to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -17,14 +17,14 @@ define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp4to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
-define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp4to16.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -36,14 +36,14 @@ define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp4to16.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
-define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp4to16.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -57,14 +57,14 @@ define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp4to16.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
-define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp4to16.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -83,14 +83,14 @@ define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp4to16.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
-define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <256 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp4to16.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -120,14 +120,14 @@ define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp4to16.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
-define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <512 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <128 x i8> %compdata) {
+; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp4to16.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<177>;
@@ -179,6 +179,6 @@ define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4)
- ret <128 x i32> %res
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.sp4to16.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4)
+ ret <512 x i8> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
index c77a43a5362513..1cf8af76cce4b0 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
@@ -4,9 +4,9 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
-define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
-; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp4to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -17,14 +17,14 @@ define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4)
- ret <2 x i32> %res
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.sp4to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ ret <8 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
-define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
-; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <16 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <8 x i8> %compdata) {
+; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp4to8.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -35,14 +35,14 @@ define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4)
- ret <4 x i32> %res
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.sp4to8.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
-define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <4 x i32> %compdata) {
-; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <32 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <16 x i8> %compdata) {
+; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp4to8.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -54,14 +54,14 @@ define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %me
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4)
- ret <8 x i32> %res
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.sp4to8.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
-define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <8 x i32> %compdata) {
-; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <64 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <32 x i8> %compdata) {
+; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp4to8.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -76,14 +76,14 @@ define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4)
- ret <16 x i32> %res
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.sp4to8.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
-define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <16 x i32> %compdata) {
-; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <128 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <64 x i8> %compdata) {
+; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp4to8.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -105,14 +105,14 @@ define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4)
- ret <32 x i32> %res
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.sp4to8.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
-define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <32 x i32> %compdata) {
-; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <256 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <128 x i8> %compdata) {
+; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp4to8.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<113>;
@@ -148,14 +148,14 @@ define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4)
- ret <64 x i32> %res
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.sp4to8.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4)
+ ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=64
-define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32> %metadata, <64 x i32> %compdata) {
-; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4)
+define <512 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32> %metadata, <256 x i8> %compdata) {
+; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp4to8.v512i8.v32i32.v256i8(<32 x i32> %metadata, <256 x i8> %compdata, /* idx_size=.b4 */ i32 4)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<225>;
@@ -219,6 +219,6 @@ define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r101, %r102, %r103, %r104};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r97, %r98, %r99, %r100};
; CHECK-NEXT: ret;
- %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, i32 8, i32 4)
- ret <128 x i32> %res
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.sp4to8.v512i8.v32i32.v256i8(<32 x i32> %metadata, <256 x i8> %compdata, i32 4)
+ ret <512 x i8> %res
}
>From 4366ff333ee0edfda74e3f9244bd3359a9e9f95e Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Fri, 11 Sep 2026 08:19:53 +0200
Subject: [PATCH 04/10] [NVVM][NVPTX] Replace spXtoY-specific forms with a
num_tgt immediate
---
llvm/docs/NVPTXUsage.md | 72 ++++---
llvm/include/llvm/IR/IntrinsicsNVVM.td | 21 +-
llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp | 50 ++---
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp | 81 +++++---
llvm/test/CodeGen/NVPTX/spcompress-invalid.ll | 24 ++-
llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll | 112 +++++-----
.../CodeGen/NVPTX/spdecompress-invalid.ll | 36 ++--
.../CodeGen/NVPTX/spdecompress-sp1to16.ll | 87 ++++----
.../test/CodeGen/NVPTX/spdecompress-sp1to2.ll | 156 +++++++-------
.../test/CodeGen/NVPTX/spdecompress-sp1to4.ll | 194 ++++++++++--------
.../test/CodeGen/NVPTX/spdecompress-sp1to8.ll | 93 +++++----
.../CodeGen/NVPTX/spdecompress-sp2to16.ll | 60 +++---
.../test/CodeGen/NVPTX/spdecompress-sp2to4.ll | 144 +++++++------
.../test/CodeGen/NVPTX/spdecompress-sp2to8.ll | 68 +++---
.../CodeGen/NVPTX/spdecompress-sp4to16.ll | 24 +--
.../test/CodeGen/NVPTX/spdecompress-sp4to8.ll | 28 +--
16 files changed, 691 insertions(+), 559 deletions(-)
diff --git a/llvm/docs/NVPTXUsage.md b/llvm/docs/NVPTXUsage.md
index 163886a0436bfc..50bf4e4434ad16 100644
--- a/llvm/docs/NVPTXUsage.md
+++ b/llvm/docs/NVPTXUsage.md
@@ -1037,34 +1037,40 @@ and `sm_107a`.
The intrinsic types representing `mdata`, `cdata`, and `data` are overloaded.
The packed metadata remains a bundle of 32-bit registers: one register is
represented by `i32`, and `N` registers are represented by `<N x i32>`. The
-`cdata` and `data` bundles instead encode the element size in their scalar type.
-Each 32-bit register is represented by four `i8` lanes or two `i16` lanes, so a
-bundle of `N` registers uses `<4N x i8>` or `<2N x i16>`, respectively. The
-actual intrinsic names include the corresponding LLVM overload suffixes.
+`cdata` and `data` vectors instead use one `i8` or `i16` lane per logical
+element. Lowering packs these lanes into 32-bit PTX register operands and pads
+an incomplete final register when necessary. The actual intrinsic names
+include the corresponding LLVM overload suffixes.
-The trailing argument is an immediate qualifier with the following encoding:
+The immediate qualifiers have the following encodings:
-| Argument | Values | PTX qualifiers |
-| ----------- | -------- | -------------- |
-| `%idx_size` | `2`, `4` | `.b2`, `.b4` |
+| Argument | Values | PTX qualifiers |
+| ----------- | --------------------- | ------------------------------- |
+| `%idx_size` | `2`, `4` | `.b2`, `.b4` |
+| `%num_tgt` | `2`, `4`, `8`, `16` | target group size in `.sp::X:Y` |
The PTX element-size qualifier (`.b8` or `.b16`) is inferred from the `i8` or
`i16` element type. The repeat-factor qualifier is inferred from the bundle
-sizes.
+sizes. `%num_tgt` is `4` for `spcompress`.
-#### '`llvm.nvvm.spcompress.sp2to4`' Intrinsic
+#### '`llvm.nvvm.spcompress`' Intrinsic
##### Syntax:
```llvm
-declare {MDataTy, CDataTy} @llvm.nvvm.spcompress.sp2to4(
- DataTy %data, i32 %spdesc, i32 immarg %idx_size)
+declare {MDataTy, CDataTy} @llvm.nvvm.spcompress(
+ DataTy %data, i32 %spdesc, i32 immarg %idx_size,
+ i32 immarg %num_tgt)
```
##### Overview:
This intrinsic compresses the dense vector `%data` using 2:4 structured
-sparsity. It returns the selected element indices as `mdata` and the selected
+sparsity. `%num_tgt` specifies the target group size and must be `4`; the
+source-to-target ratio is inferred from the `CDataTy` and `DataTy` vector
+lengths. In `.sp::X:Y`, `Y` is `%num_tgt` and
+`X = Y * num_elements(CDataTy) / num_elements(DataTy)`; the division must be
+exact. It returns the selected element indices as `mdata` and the selected
elements as `cdata`. The number of 32-bit registers in each bundle is:
| Bundle | Register count |
@@ -1100,27 +1106,35 @@ the selected indices are implementation-specific.
For more information, see the
[PTX ISA](https://docs.nvidia.com/cuda/parallel-thread-execution/#data-movement-and-conversion-instructions-spcompress).
-#### '`llvm.nvvm.spdecompress.*`' Intrinsics
+#### '`llvm.nvvm.spdecompress`' Intrinsic
##### Syntax:
```llvm
-declare DataTy @llvm.nvvm.spdecompress.{sp1to2,sp1to4,sp1to8,sp1to16,
- sp2to4,sp2to8,sp2to16,
- sp4to8,sp4to16}(
- MDataTy %mdata, CDataTy %cdata, i32 immarg %idx_size)
+declare DataTy @llvm.nvvm.spdecompress(
+ MDataTy %mdata, CDataTy %cdata, i32 immarg %idx_size,
+ i32 immarg %num_tgt)
```
##### Overview:
-These intrinsics decompress the structured sparse vector `%cdata` into a
-dense `data` vector. The intrinsic name specifies the number of source and
-target elements in each group. For example, `sp2to4` maps two compressed
-elements into a group of four dense elements using the indices in `%mdata`.
-Dense positions not selected by the metadata are set to zero.
+This intrinsic decompresses the structured sparse vector `%cdata` into a dense
+`data` vector. `%num_tgt` specifies `Y`, the number of dense elements in each
+target group. The source group size `X` in `.sp::X:Y` is calculated from the
+vector lengths as:
-For an intrinsic named `spXtoY`, the number of 32-bit registers in each bundle
-is:
+`X = Y * num_elements(CDataTy) / num_elements(DataTy)`
+
+The division must be exact. For example, two compressed lanes for every four
+result lanes with `%num_tgt = 4` select `.sp::2:4`. Dense positions not selected
+by the metadata are set to zero.
+
+Let `Y` be `%num_tgt`. The repeat factor `num` and source group size `X` are:
+
+- `num = num_elements(DataTy) / Y`.
+- `X = num_elements(CDataTy) / num`.
+
+The number of 32-bit PTX registers in each bundle is:
| Bundle | Register count |
| ------- | ------------------------------------------------- |
@@ -1129,12 +1143,14 @@ is:
| `data` | `ceil(Y * elem_size * num / 32)` |
Here, `elem_size` is the common `i8` or `i16` scalar bit width of `CDataTy` and
-`DataTy`. `num` is inferred from the number of registers in `DataTy` and
-determines the PTX repeat-factor qualifier (`.x1`, `.x2`, ..., `.x64`). Any
-unused lanes in the final `CDataTy` or `DataTy` register are padding.
+`DataTy`. `num` determines the PTX repeat-factor qualifier (`.x1`, `.x2`, ...,
+`.x64`). Padding needed to fill the final compressed-data register is added
+internally during lowering and is not represented in `CDataTy`.
The following conditions must hold:
+- `X:Y` is one of `1:2`, `1:4`, `1:8`, `1:16`, `2:4`, `2:8`, `2:16`,
+ `4:8`, or `4:16`; consequently, `X < Y`.
- `X * elem_size <= 32`.
- `%idx_size` is `2` only when `Y <= 4`.
- `32 <= Y * elem_size * num <= 4096`.
diff --git a/llvm/include/llvm/IR/IntrinsicsNVVM.td b/llvm/include/llvm/IR/IntrinsicsNVVM.td
index 45d117586d811e..380341b352d9b2 100644
--- a/llvm/include/llvm/IR/IntrinsicsNVVM.td
+++ b/llvm/include/llvm/IR/IntrinsicsNVVM.td
@@ -4236,11 +4236,14 @@ class SPCompress :
llvm_any_vector_int_ty], // cdata
[llvm_any_vector_int_ty, // data
llvm_i32_ty], // spdesc
- [llvm_i32_ty], // idx_size
+ [llvm_i32_ty, // idx_size
+ llvm_i32_ty], // num_tgt
!listconcat([IntrNoMem, IntrSpeculatable],
- NVVM_SP_IDX_ARGPROPS<ArgIndex<2>>.Prop)>;
+ NVVM_SP_IDX_ARGPROPS<ArgIndex<2>>.Prop,
+ [RangeList<ArgIndex<3>, [4]>,
+ ArgInfo<ArgIndex<3>, [ArgName<"num_tgt">]>])>;
-def int_nvvm_spcompress_sp2to4 : SPCompress;
+def int_nvvm_spcompress : SPCompress;
//
// spdecompress intrinsics
@@ -4251,13 +4254,13 @@ class SPDecompress :
[llvm_any_vector_int_ty], // data
[llvm_anyint_ty, // mdata
llvm_any_vector_int_ty], // cdata
- [llvm_i32_ty], // idx_size
+ [llvm_i32_ty, // idx_size
+ llvm_i32_ty], // num_tgt
!listconcat([IntrNoMem, IntrSpeculatable],
- NVVM_SP_IDX_ARGPROPS<ArgIndex<2>>.Prop)>;
+ NVVM_SP_IDX_ARGPROPS<ArgIndex<2>>.Prop,
+ [RangeList<ArgIndex<3>, [2, 4, 8, 16]>,
+ ArgInfo<ArgIndex<3>, [ArgName<"num_tgt">]>])>;
-foreach num_src = [1, 2, 4] in
- foreach num_tgt = [2, 4, 8, 16] in
- if !lt(num_src, num_tgt) then
- def int_nvvm_spdecompress_sp # num_src # to # num_tgt : SPDecompress;
+def int_nvvm_spdecompress : SPDecompress;
} // let TargetPrefix = "nvvm"
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
index 966ad3c171fabe..e085da8f0c5e7d 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
@@ -140,7 +140,7 @@ class NVPTXDAGToDAGISel : public SelectionDAGISel {
void SelectTcgen05Ld(SDNode *N, bool hasOffset = false);
void SelectTcgen05St(SDNode *N, bool hasOffset = false);
void selectSPCompress(SDNode *N);
- void selectSPDecompress(SDNode *N, unsigned NumSrc, unsigned NumTgt);
+ void selectSPDecompress(SDNode *N);
void selectAtomicSwap128(SDNode *N);
inline SDValue getI32Imm(unsigned Imm, const SDLoc &DL) {
@@ -489,13 +489,18 @@ static std::optional<unsigned> getSPDataElemSize(EVT VT) {
CASE_MACRO(128)
void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
- constexpr unsigned NumFlags = 1;
+ constexpr unsigned NumFlags = 2;
if (N->getNumOperands() < NumFlags + 3)
report_fatal_error("Malformed spcompress intrinsic");
unsigned FlagOp = N->getNumOperands() - NumFlags;
unsigned IdxSize =
cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
+ unsigned NumTgt =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
+ if (NumTgt != 4)
+ report_fatal_error(Twine("Invalid spcompress flags: num_tgt=") +
+ Twine(NumTgt));
// The data bundle contains two registers per repetition.
unsigned DataSize = FlagOp - 2;
@@ -558,15 +563,20 @@ void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
#undef REPLACE_SPCOMPRESS_NODE
}
-void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
- unsigned NumTgt) {
- constexpr unsigned NumFlags = 1;
+void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N) {
+ // NumSrc is inferred from the logical vector lengths before register
+ // padding and appended as an internal operand by lowerSPDecompress.
+ constexpr unsigned NumFlags = 3;
if (N->getNumOperands() < NumFlags + 3)
report_fatal_error("Malformed spdecompress intrinsic");
unsigned FlagOp = N->getNumOperands() - NumFlags;
unsigned IdxSize =
cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
+ unsigned NumTgt =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
+ unsigned NumSrc =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 2))->getZExtValue();
EVT DataRegVT = N->getValueType(0);
auto ElemSize = getSPDataElemSize(DataRegVT);
if (!ElemSize)
@@ -643,35 +653,11 @@ bool NVPTXDAGToDAGISel::tryIntrinsicNoChain(SDNode *N) {
switch (IID) {
default:
return false;
- case Intrinsic::nvvm_spcompress_sp2to4:
+ case Intrinsic::nvvm_spcompress:
selectSPCompress(N);
return true;
- case Intrinsic::nvvm_spdecompress_sp1to2:
- selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/2);
- return true;
- case Intrinsic::nvvm_spdecompress_sp1to4:
- selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/4);
- return true;
- case Intrinsic::nvvm_spdecompress_sp1to8:
- selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/8);
- return true;
- case Intrinsic::nvvm_spdecompress_sp1to16:
- selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/16);
- return true;
- case Intrinsic::nvvm_spdecompress_sp2to4:
- selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/4);
- return true;
- case Intrinsic::nvvm_spdecompress_sp2to8:
- selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/8);
- return true;
- case Intrinsic::nvvm_spdecompress_sp2to16:
- selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/16);
- return true;
- case Intrinsic::nvvm_spdecompress_sp4to8:
- selectSPDecompress(N, /*NumSrc=*/4, /*NumTgt=*/8);
- return true;
- case Intrinsic::nvvm_spdecompress_sp4to16:
- selectSPDecompress(N, /*NumSrc=*/4, /*NumTgt=*/16);
+ case Intrinsic::nvvm_spdecompress:
+ selectSPDecompress(N);
return true;
}
}
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
index f3a0e943605997..0700eb841ffe4b 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
@@ -65,6 +65,7 @@
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/ErrorHandling.h"
#include "llvm/Support/KnownBits.h"
+#include "llvm/Support/MathExtras.h"
#include "llvm/Support/NVPTXAddrSpace.h"
#include "llvm/Target/TargetMachine.h"
#include "llvm/Target/TargetOptions.h"
@@ -3215,10 +3216,7 @@ static EVT getSPVectorPartType(EVT VT) {
static unsigned getSPVectorNumParts(EVT VT) {
EVT PartVT = getSPVectorPartType(VT);
unsigned PartElts = PartVT.isVector() ? PartVT.getVectorNumElements() : 1;
- if (VT.getVectorNumElements() % PartElts != 0)
- report_fatal_error(
- "SP intrinsic vector types must occupy whole 32-bit registers");
- return VT.getVectorNumElements() / PartElts;
+ return divideCeil(VT.getVectorNumElements(), PartElts);
}
static bool needsSPVectorLowering(SDNode *N) {
@@ -3243,6 +3241,14 @@ static void splitSPVector(SDValue Vector, SmallVectorImpl<SDValue> &Parts,
unsigned NumParts = getSPVectorNumParts(VT);
unsigned PartElts = PartVT.getVectorNumElements();
+ unsigned PaddedElts = NumParts * PartElts;
+ if (PaddedElts != VT.getVectorNumElements()) {
+ EVT PaddedVT = EVT::getVectorVT(*DAG.getContext(),
+ VT.getVectorElementType(), PaddedElts);
+ Vector = DAG.getNode(ISD::INSERT_SUBVECTOR, SDLoc(Vector), PaddedVT,
+ DAG.getUNDEF(PaddedVT), Vector,
+ DAG.getVectorIdxConstant(0, SDLoc(Vector)));
+ }
for (unsigned I = 0; I != NumParts; ++I)
Parts.push_back(
DAG.getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(Vector), PartVT, Vector,
@@ -3255,8 +3261,19 @@ static SDValue joinSPVector(EVT VT, ArrayRef<SDValue> Parts, const SDLoc &DL,
"incorrect number of SP vector parts");
if (Parts.size() == 1 && Parts.front().getValueType() == VT)
return Parts.front();
- if (Parts.front().getValueType().isVector())
- return DAG.getNode(ISD::CONCAT_VECTORS, DL, VT, Parts);
+ if (Parts.front().getValueType().isVector()) {
+ EVT PartVT = Parts.front().getValueType();
+ EVT JoinedVT =
+ EVT::getVectorVT(*DAG.getContext(), VT.getVectorElementType(),
+ Parts.size() * PartVT.getVectorNumElements());
+ SDValue Joined = Parts.size() == 1 ? Parts.front()
+ : DAG.getNode(ISD::CONCAT_VECTORS, DL,
+ JoinedVT, Parts);
+ if (JoinedVT == VT)
+ return Joined;
+ return DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, VT, Joined,
+ DAG.getVectorIdxConstant(0, DL));
+ }
return DAG.getBuildVector(VT, DL, Parts);
}
@@ -3304,10 +3321,29 @@ static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
static SDValue lowerSPDecompress(SDValue Op, SelectionDAG &DAG) {
SDNode *N = Op.getNode();
SDLoc DL(N);
- EVT ResVT0 = N->getValueType(0);
- if (!needsSPVectorLowering(N))
+ // A lowered node has the inferred num_src operand appended and may have more
+ // than one operand for each register bundle.
+ if (N->getNumOperands() != 5)
return Op;
+ EVT ResVT0 = N->getValueType(0);
+ EVT CDataVT = N->getOperand(2).getValueType();
+ if (!ResVT0.isVector() || !CDataVT.isVector())
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
+
+ unsigned NumTgt = cast<ConstantSDNode>(N->getOperand(4))->getZExtValue();
+ unsigned DataElts = ResVT0.getVectorNumElements();
+ if (NumTgt == 0 || DataElts % NumTgt != 0)
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
+ unsigned RepeatFactor = DataElts / NumTgt;
+ unsigned CDataElts = CDataVT.getVectorNumElements();
+ if (RepeatFactor == 0 || CDataElts % RepeatFactor != 0)
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
+ unsigned NumSrc = CDataElts / RepeatFactor;
+
unsigned NumParts = getSPVectorNumParts(ResVT0);
EVT PartVT = getSPVectorPartType(ResVT0);
SmallVector<EVT, 16> ListVTs(NumParts, PartVT);
@@ -3323,6 +3359,7 @@ static SDValue lowerSPDecompress(SDValue Op, SelectionDAG &DAG) {
else
Ops.push_back(Operand);
}
+ Ops.push_back(DAG.getConstant(NumSrc, DL, MVT::i32));
SDValue NewNode = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResVTs, Ops);
@@ -3361,18 +3398,10 @@ static SDValue lowerIntrinsicWOChain(SDValue Op, SelectionDAG &DAG) {
case Intrinsic::nvvm_f32x4_to_e2m1x4_rs_relu_satfinite:
return lowerCvtRSIntrinsics(Op, DAG);
- case Intrinsic::nvvm_spcompress_sp2to4:
+ case Intrinsic::nvvm_spcompress:
return lowerSPCompress(Op, DAG);
- case Intrinsic::nvvm_spdecompress_sp1to2:
- case Intrinsic::nvvm_spdecompress_sp1to4:
- case Intrinsic::nvvm_spdecompress_sp1to8:
- case Intrinsic::nvvm_spdecompress_sp1to16:
- case Intrinsic::nvvm_spdecompress_sp2to4:
- case Intrinsic::nvvm_spdecompress_sp2to8:
- case Intrinsic::nvvm_spdecompress_sp2to16:
- case Intrinsic::nvvm_spdecompress_sp4to8:
- case Intrinsic::nvvm_spdecompress_sp4to16:
+ case Intrinsic::nvvm_spdecompress:
return lowerSPDecompress(Op, DAG);
}
}
@@ -7437,20 +7466,14 @@ static SDValue combineIntrinsicWOChain(SDNode *N,
return diagnoseUnsupportedFAdd(N, DCI.DAG, IID, RoundingMode);
return combineFAddWithNeg(N, DCI.DAG, IID, RoundingMode);
}
- case Intrinsic::nvvm_spcompress_sp2to4:
+ case Intrinsic::nvvm_spcompress:
if (needsSPVectorLowering(N))
return lowerSPCompress(SDValue(N, 0), DCI.DAG);
break;
- case Intrinsic::nvvm_spdecompress_sp1to2:
- case Intrinsic::nvvm_spdecompress_sp1to4:
- case Intrinsic::nvvm_spdecompress_sp1to8:
- case Intrinsic::nvvm_spdecompress_sp1to16:
- case Intrinsic::nvvm_spdecompress_sp2to4:
- case Intrinsic::nvvm_spdecompress_sp2to8:
- case Intrinsic::nvvm_spdecompress_sp2to16:
- case Intrinsic::nvvm_spdecompress_sp4to8:
- case Intrinsic::nvvm_spdecompress_sp4to16: {
- if (needsSPVectorLowering(N))
+ case Intrinsic::nvvm_spdecompress: {
+ // The IR intrinsic has the ID plus four arguments; lowered nodes also have
+ // an inferred num_src operand, so do not lower them again.
+ if (N->getNumOperands() == 5)
return lowerSPDecompress(SDValue(N, 0), DCI.DAG);
break;
}
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
index 6f80f646951fe9..fd4340338f9913 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
@@ -1,6 +1,7 @@
; RUN: split-file %s %t
; RUN: not --crash llc < %t/bad-elem-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
+; RUN: not llvm-as -disable-output < %t/bad-num-tgt.ll 2>&1 | FileCheck %s --check-prefix=BAD-NUM-TGT
; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
@@ -12,7 +13,7 @@
define { i32, <2 x i32> } @bad_spcompress_elem_type(i32 %spdesc,
<2 x i32> %data) {
; BAD-ELEM: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 2)
+ %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.i32.v2i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 2, i32 4)
ret { i32, <2 x i32> } %res
}
@@ -21,7 +22,16 @@ define { i32, <2 x i32> } @bad_spcompress_elem_type(i32 %spdesc,
define { i32, <4 x i8> } @bad_spcompress_idx_size(i32 %spdesc,
<8 x i8> %data) {
; BAD-IDX: immarg value 3 for arg 2 out of range set
- %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 3)
+ %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 3, i32 4)
+ ret { i32, <4 x i8> } %res
+}
+
+;--- bad-num-tgt.ll
+
+define { i32, <4 x i8> } @bad_spcompress_num_tgt(i32 %spdesc,
+ <8 x i8> %data) {
+; BAD-NUM-TGT: immarg value 8 for arg 3 out of range set
+ %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2, i32 8)
ret { i32, <4 x i8> } %res
}
@@ -30,7 +40,7 @@ define { i32, <4 x i8> } @bad_spcompress_idx_size(i32 %spdesc,
define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
<32 x i8> %data) {
; BAD-MDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4)
+ %res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4, i32 4)
%mdata = extractvalue { i32, <16 x i8> } %res, 0
ret i32 %mdata
}
@@ -40,7 +50,7 @@ define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
define <8 x i8> @bad_spcompress_compressed_data_register_count(
i32 %spdesc, <32 x i8> %data) {
; BAD-CDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { <2 x i32>, <8 x i8> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4)
+ %res = call { <2 x i32>, <8 x i8> } @llvm.nvvm.spcompress.v2i32.v8i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4, i32 4)
%cdata = extractvalue { <2 x i32>, <8 x i8> } %res, 1
ret <8 x i8> %cdata
}
@@ -50,7 +60,7 @@ define <8 x i8> @bad_spcompress_compressed_data_register_count(
define <16 x i8> @bad_spcompress_data_register_count(i32 %spdesc,
<24 x i8> %data) {
; BAD-DATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i8.v24i8(<24 x i8> %data, i32 %spdesc, i32 4)
+ %res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.v2i32.v16i8.v24i8(<24 x i8> %data, i32 %spdesc, i32 4, i32 4)
%cdata = extractvalue { <2 x i32>, <16 x i8> } %res, 1
ret <16 x i8> %cdata
}
@@ -59,7 +69,7 @@ define <16 x i8> @bad_spcompress_data_register_count(i32 %spdesc,
define i64 @bad_spcompress_result_type(i32 %spdesc, <8 x i8> %data) {
; BAD-RESULT-TY: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { i64, <4 x i8> } @llvm.nvvm.spcompress.sp2to4.i64.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2)
+ %res = call { i64, <4 x i8> } @llvm.nvvm.spcompress.i64.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2, i32 4)
%mdata = extractvalue { i64, <4 x i8> } %res, 0
ret i64 %mdata
}
@@ -69,6 +79,6 @@ define i64 @bad_spcompress_result_type(i32 %spdesc, <8 x i8> %data) {
define { i32, <2 x i16> } @bad_spcompress_data_type(i32 %spdesc,
<8 x i8> %data) {
; BAD-DATA-TY: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
- %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v8i8(<8 x i8> %data, i32 %spdesc, i32 2)
+ %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.i32.v2i16.v8i8(<8 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret { i32, <2 x i16> } %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
index ff19ab469fa4ac..23d9d7a8d2b9bc 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
@@ -5,7 +5,7 @@
define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <8 x i8> %data) {
; FORMAT-LABEL: define { i32, <4 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -17,13 +17,13 @@ define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <8
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2)
+ %res = call {i32, <4 x i8>} @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret {i32, <4 x i8>} %res
}
define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <4 x i16> %data) {
; FORMAT-LABEL: define { i32, <2 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -35,13 +35,13 @@ define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, i32 2)
+ %res = call {i32, <2 x i16>} @llvm.nvvm.spcompress.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, i32 2, i32 4)
ret {i32, <2 x i16>} %res
}
define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <8 x i8> %data) {
; FORMAT-LABEL: define { i32, <4 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -53,13 +53,13 @@ define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <8
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 4)
+ %res = call {i32, <4 x i8>} @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 4, i32 4)
ret {i32, <4 x i8>} %res
}
define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <4 x i16> %data) {
; FORMAT-LABEL: define { i32, <2 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -71,13 +71,13 @@ define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <
; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
; CHECK-NEXT: ret;
- %res = call {i32, <2 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, i32 4)
+ %res = call {i32, <2 x i16>} @llvm.nvvm.spcompress.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, i32 4, i32 4)
ret {i32, <2 x i16>} %res
}
define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <16 x i8> %data) {
; FORMAT-LABEL: define { i32, <8 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -89,13 +89,13 @@ define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <16
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <8 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, i32 2)
+ %res = call {i32, <8 x i8>} @llvm.nvvm.spcompress.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret {i32, <8 x i8>} %res
}
define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <8 x i16> %data) {
; FORMAT-LABEL: define { i32, <4 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -107,13 +107,13 @@ define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, i32 2)
+ %res = call {i32, <4 x i16>} @llvm.nvvm.spcompress.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, i32 2, i32 4)
ret {i32, <4 x i16>} %res
}
define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <16 x i8> %data) {
; FORMAT-LABEL: define { i32, <8 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -125,13 +125,13 @@ define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <16
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <8 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, i32 4)
+ %res = call {i32, <8 x i8>} @llvm.nvvm.spcompress.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, i32 4, i32 4)
ret {i32, <8 x i8>} %res
}
define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <8 x i16> %data) {
; FORMAT-LABEL: define { i32, <4 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -143,13 +143,13 @@ define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
; CHECK-NEXT: ret;
- %res = call {i32, <4 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, i32 4)
+ %res = call {i32, <4 x i16>} @llvm.nvvm.spcompress.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, i32 4, i32 4)
ret {i32, <4 x i16>} %res
}
define {i32, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <32 x i8> %data) {
; FORMAT-LABEL: define { i32, <16 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.sp2to4.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -162,13 +162,13 @@ define {i32, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <3
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <16 x i8>} @llvm.nvvm.spcompress.sp2to4.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 2)
+ %res = call {i32, <16 x i8>} @llvm.nvvm.spcompress.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret {i32, <16 x i8>} %res
}
define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <16 x i16> %data) {
; FORMAT-LABEL: define { i32, <8 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -181,13 +181,13 @@ define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <8 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, i32 2)
+ %res = call {i32, <8 x i16>} @llvm.nvvm.spcompress.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, i32 2, i32 4)
ret {i32, <8 x i16>} %res
}
define {<2 x i32>, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spdesc, <32 x i8> %data) {
; FORMAT-LABEL: define { <2 x i32>, <16 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.v2i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<16>;
@@ -200,13 +200,13 @@ define {<2 x i32>, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r12, %r13, %r14, %r15};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r10, %r11};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <16 x i8>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4)
+ %res = call {<2 x i32>, <16 x i8>} @llvm.nvvm.spcompress.v2i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4, i32 4)
ret {<2 x i32>, <16 x i8>} %res
}
define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <16 x i16> %data) {
; FORMAT-LABEL: define { i32, <8 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -219,13 +219,13 @@ define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
; CHECK-NEXT: ret;
- %res = call {i32, <8 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, i32 4)
+ %res = call {i32, <8 x i16>} @llvm.nvvm.spcompress.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, i32 4, i32 4)
ret {i32, <8 x i16>} %res
}
define {<2 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spdesc, <64 x i8> %data) {
; FORMAT-LABEL: define { <2 x i32>, <32 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i8> } @llvm.nvvm.spcompress.sp2to4.v2i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i8> } @llvm.nvvm.spcompress.v2i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<28>;
@@ -241,13 +241,13 @@ define {<2 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <32 x i8>} @llvm.nvvm.spcompress.sp2to4.v2i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, i32 2)
+ %res = call {<2 x i32>, <32 x i8>} @llvm.nvvm.spcompress.v2i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret {<2 x i32>, <32 x i8>} %res
}
define {i32, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <32 x i16> %data) {
; FORMAT-LABEL: define { i32, <16 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call { i32, <16 x i16> } @llvm.nvvm.spcompress.sp2to4.i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { i32, <16 x i16> } @llvm.nvvm.spcompress.i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -263,13 +263,13 @@ define {i32, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc,
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r18;
; CHECK-NEXT: ret;
- %res = call {i32, <16 x i16>} @llvm.nvvm.spcompress.sp2to4.i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, i32 2)
+ %res = call {i32, <16 x i16>} @llvm.nvvm.spcompress.i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, i32 2, i32 4)
ret {i32, <16 x i16>} %res
}
define {<4 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spdesc, <64 x i8> %data) {
; FORMAT-LABEL: define { <4 x i32>, <32 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i8> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i8> } @llvm.nvvm.spcompress.v4i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<30>;
@@ -285,13 +285,13 @@ define {<4 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spde
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r22, %r23, %r24, %r25};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r18, %r19, %r20, %r21};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <32 x i8>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, i32 4)
+ %res = call {<4 x i32>, <32 x i8>} @llvm.nvvm.spcompress.v4i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, i32 4, i32 4)
ret {<4 x i32>, <32 x i8>} %res
}
define {<2 x i32>, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spdesc, <32 x i16> %data) {
; FORMAT-LABEL: define { <2 x i32>, <16 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i16> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i16> } @llvm.nvvm.spcompress.v2i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<28>;
@@ -307,13 +307,13 @@ define {<2 x i32>, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %sp
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <16 x i16>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, i32 4)
+ %res = call {<2 x i32>, <16 x i16>} @llvm.nvvm.spcompress.v2i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, i32 4, i32 4)
ret {<2 x i32>, <16 x i16>} %res
}
define {<4 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spdesc, <128 x i8> %data) {
; FORMAT-LABEL: define { <4 x i32>, <64 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i8> } @llvm.nvvm.spcompress.sp2to4.v4i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i8> } @llvm.nvvm.spcompress.v4i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<54>;
@@ -335,13 +335,13 @@ define {<4 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spd
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <64 x i8>} @llvm.nvvm.spcompress.sp2to4.v4i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, i32 2)
+ %res = call {<4 x i32>, <64 x i8>} @llvm.nvvm.spcompress.v4i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret {<4 x i32>, <64 x i8>} %res
}
define {<2 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %spdesc, <64 x i16> %data) {
; FORMAT-LABEL: define { <2 x i32>, <32 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i16> } @llvm.nvvm.spcompress.sp2to4.v2i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i16> } @llvm.nvvm.spcompress.v2i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<52>;
@@ -363,13 +363,13 @@ define {<2 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r36, %r37, %r38, %r39};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r34, %r35};
; CHECK-NEXT: ret;
- %res = call {<2 x i32>, <32 x i16>} @llvm.nvvm.spcompress.sp2to4.v2i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, i32 2)
+ %res = call {<2 x i32>, <32 x i16>} @llvm.nvvm.spcompress.v2i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, i32 2, i32 4)
ret {<2 x i32>, <32 x i16>} %res
}
define {<8 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spdesc, <128 x i8> %data) {
; FORMAT-LABEL: define { <8 x i32>, <64 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i8> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i8> } @llvm.nvvm.spcompress.v8i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<58>;
@@ -392,13 +392,13 @@ define {<8 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spd
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <64 x i8>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, i32 4)
+ %res = call {<8 x i32>, <64 x i8>} @llvm.nvvm.spcompress.v8i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, i32 4, i32 4)
ret {<8 x i32>, <64 x i8>} %res
}
define {<4 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %spdesc, <64 x i16> %data) {
; FORMAT-LABEL: define { <4 x i32>, <32 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i16> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i16> } @llvm.nvvm.spcompress.v4i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<54>;
@@ -420,13 +420,13 @@ define {<4 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <32 x i16>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, i32 4)
+ %res = call {<4 x i32>, <32 x i16>} @llvm.nvvm.spcompress.v4i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, i32 4, i32 4)
ret {<4 x i32>, <32 x i16>} %res
}
define {<8 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %spdesc, <256 x i8> %data) {
; FORMAT-LABEL: define { <8 x i32>, <128 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i8> } @llvm.nvvm.spcompress.sp2to4.v8i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i8> } @llvm.nvvm.spcompress.v8i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<106>;
@@ -461,13 +461,13 @@ define {<8 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %sp
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <128 x i8>} @llvm.nvvm.spcompress.sp2to4.v8i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, i32 2)
+ %res = call {<8 x i32>, <128 x i8>} @llvm.nvvm.spcompress.v8i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret {<8 x i32>, <128 x i8>} %res
}
define {<4 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %spdesc, <128 x i16> %data) {
; FORMAT-LABEL: define { <4 x i32>, <64 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i16> } @llvm.nvvm.spcompress.sp2to4.v4i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i16> } @llvm.nvvm.spcompress.v4i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<102>;
@@ -501,13 +501,13 @@ define {<4 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<4 x i32>, <64 x i16>} @llvm.nvvm.spcompress.sp2to4.v4i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, i32 2)
+ %res = call {<4 x i32>, <64 x i16>} @llvm.nvvm.spcompress.v4i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, i32 2, i32 4)
ret {<4 x i32>, <64 x i16>} %res
}
define {<16 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %spdesc, <256 x i8> %data) {
; FORMAT-LABEL: define { <16 x i32>, <128 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i8> } @llvm.nvvm.spcompress.sp2to4.v16i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i8> } @llvm.nvvm.spcompress.v16i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<114>;
@@ -544,13 +544,13 @@ define {<16 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <128 x i8>} @llvm.nvvm.spcompress.sp2to4.v16i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, i32 4)
+ %res = call {<16 x i32>, <128 x i8>} @llvm.nvvm.spcompress.v16i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, i32 4, i32 4)
ret {<16 x i32>, <128 x i8>} %res
}
define {<8 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %spdesc, <128 x i16> %data) {
; FORMAT-LABEL: define { <8 x i32>, <64 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i16> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i16> } @llvm.nvvm.spcompress.v8i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<106>;
@@ -585,13 +585,13 @@ define {<8 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <64 x i16>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, i32 4)
+ %res = call {<8 x i32>, <64 x i16>} @llvm.nvvm.spcompress.v8i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, i32 4, i32 4)
ret {<8 x i32>, <64 x i16>} %res
}
define {<16 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %spdesc, <512 x i8> %data) {
; FORMAT-LABEL: define { <16 x i32>, <256 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call { <16 x i32>, <256 x i8> } @llvm.nvvm.spcompress.sp2to4.v16i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { <16 x i32>, <256 x i8> } @llvm.nvvm.spcompress.v16i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<210>;
@@ -652,13 +652,13 @@ define {<16 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <256 x i8>} @llvm.nvvm.spcompress.sp2to4.v16i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, i32 2)
+ %res = call {<16 x i32>, <256 x i8>} @llvm.nvvm.spcompress.v16i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret {<16 x i32>, <256 x i8>} %res
}
define {<8 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %spdesc, <256 x i16> %data) {
; FORMAT-LABEL: define { <8 x i32>, <128 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i16> } @llvm.nvvm.spcompress.sp2to4.v8i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i16> } @llvm.nvvm.spcompress.v8i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<202>;
@@ -717,13 +717,13 @@ define {<8 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<8 x i32>, <128 x i16>} @llvm.nvvm.spcompress.sp2to4.v8i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, i32 2)
+ %res = call {<8 x i32>, <128 x i16>} @llvm.nvvm.spcompress.v8i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, i32 2, i32 4)
ret {<8 x i32>, <128 x i16>} %res
}
define {<32 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %spdesc, <512 x i8> %data) {
; FORMAT-LABEL: define { <32 x i32>, <256 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call { <32 x i32>, <256 x i8> } @llvm.nvvm.spcompress.sp2to4.v32i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <32 x i32>, <256 x i8> } @llvm.nvvm.spcompress.v32i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<226>;
@@ -788,13 +788,13 @@ define {<32 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %s
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<32 x i32>, <256 x i8>} @llvm.nvvm.spcompress.sp2to4.v32i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, i32 4)
+ %res = call {<32 x i32>, <256 x i8>} @llvm.nvvm.spcompress.v32i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, i32 4, i32 4)
ret {<32 x i32>, <256 x i8>} %res
}
define {<16 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %spdesc, <256 x i16> %data) {
; FORMAT-LABEL: define { <16 x i32>, <128 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i16> } @llvm.nvvm.spcompress.sp2to4.v16i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i16> } @llvm.nvvm.spcompress.v16i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<210>;
@@ -855,6 +855,6 @@ define {<16 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
; CHECK-NEXT: ret;
- %res = call {<16 x i32>, <128 x i16>} @llvm.nvvm.spcompress.sp2to4.v16i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, i32 4)
+ %res = call {<16 x i32>, <128 x i16>} @llvm.nvvm.spcompress.v16i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, i32 4, i32 4)
ret {<16 x i32>, <128 x i16>} %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
index 07f494ac7b58f5..d4e1b743363875 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
@@ -1,6 +1,7 @@
; RUN: split-file %s %t
; RUN: not --crash llc < %t/bad-elem-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
+; RUN: not llvm-as -disable-output < %t/bad-num-tgt.ll 2>&1 | FileCheck %s --check-prefix=BAD-NUM-TGT
; RUN: not --crash llc < %t/bad-idx-size-for-target.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-IDX-TARGET
; RUN: not --crash llc < %t/bad-input-span.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-INPUT-SPAN
; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
@@ -14,7 +15,7 @@
define <2 x i32> @bad_spdecompress_elem_type(i32 %metadata,
<2 x i32> %compdata) {
; BAD-ELEM: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 2)
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.v2i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 2, i32 4)
ret <2 x i32> %res
}
@@ -23,7 +24,16 @@ define <2 x i32> @bad_spdecompress_elem_type(i32 %metadata,
define <4 x i8> @bad_spdecompress_idx_size(i32 %metadata,
<4 x i8> %compdata) {
; BAD-IDX: immarg value 3 for arg 2 out of range set
- %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 3)
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 3, i32 4)
+ ret <4 x i8> %res
+}
+
+;--- bad-num-tgt.ll
+
+define <4 x i8> @bad_spdecompress_num_tgt(i32 %metadata,
+ <2 x i8> %compdata) {
+; BAD-NUM-TGT: immarg value 3 for arg 3 out of range set
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 3)
ret <4 x i8> %res
}
@@ -32,7 +42,7 @@ define <4 x i8> @bad_spdecompress_idx_size(i32 %metadata,
define <32 x i8> @bad_spdecompress_idx_size_for_target(
i32 %metadata, <4 x i8> %compdata) {
; BAD-IDX-TARGET: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=8, elem_size=8, idx_size=2
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to8.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2, i32 8)
ret <32 x i8> %res
}
@@ -41,7 +51,7 @@ define <32 x i8> @bad_spdecompress_idx_size_for_target(
define <8 x i16> @bad_spdecompress_input_span(<2 x i32> %metadata,
<4 x i16> %compdata) {
; BAD-INPUT-SPAN: LLVM ERROR: Invalid spdecompress flags: num_src=4, num_tgt=8, elem_size=16, idx_size=4
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp4to8.v8i16.v2i32.v4i16(<2 x i32> %metadata, <4 x i16> %compdata, i32 4)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.v2i32.v4i16(<2 x i32> %metadata, <4 x i16> %compdata, i32 4, i32 8)
ret <8 x i16> %res
}
@@ -50,16 +60,16 @@ define <8 x i16> @bad_spdecompress_input_span(<2 x i32> %metadata,
define <16 x i8> @bad_spdecompress_metadata_register_count(
<2 x i32> %metadata, <8 x i8> %compdata) {
; BAD-MDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.v2i32.v8i8(<2 x i32> %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.v2i32.v8i8(<2 x i32> %metadata, <8 x i8> %compdata, i32 4, i32 4)
ret <16 x i8> %res
}
;--- bad-cdata-size.ll
define <8 x i16> @bad_spdecompress_compressed_data_register_count(
- i32 %metadata, <2 x i16> %compdata) {
-; BAD-CDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ i32 %metadata, <6 x i16> %compdata) {
+; BAD-CDATA: LLVM ERROR: Invalid spdecompress flags: num_src=3, num_tgt=4, elem_size=16, idx_size=4
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v6i16(i32 %metadata, <6 x i16> %compdata, i32 4, i32 4)
ret <8 x i16> %res
}
@@ -68,24 +78,24 @@ define <8 x i16> @bad_spdecompress_compressed_data_register_count(
define <12 x i8> @bad_spdecompress_data_register_count(
i32 %metadata, <8 x i8> %compdata) {
; BAD-DATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <12 x i8> @llvm.nvvm.spdecompress.sp2to4.v12i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <12 x i8> @llvm.nvvm.spdecompress.v12i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 4)
ret <12 x i8> %res
}
;--- bad-result-type.ll
define <4 x i16> @bad_spdecompress_result_type(i32 %metadata,
- <4 x i8> %compdata) {
+ <2 x i8> %compdata) {
; BAD-RESULT-TY: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 4)
ret <4 x i16> %res
}
;--- bad-mdata-type.ll
define <4 x i8> @bad_spdecompress_metadata_type(i64 %metadata,
- <4 x i8> %compdata) {
+ <2 x i8> %compdata) {
; BAD-MDATA-TY: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
- %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i64.v4i8(i64 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i64.v2i8(i64 %metadata, <2 x i8> %compdata, i32 2, i32 4)
ret <4 x i8> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
index 6359c22a8dbf72..15aec00fce07a5 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
@@ -4,44 +4,53 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4
-define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <1 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-NEXT: .reg .b32 %r<13>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ld.param::func.b8 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: prmt.b32 %r2, %r1, %r3, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r4, %r5, %r6, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r7, %r2, %r4, 0x5410U;
+; CHECK-NEXT: ld.param::func.b32 %r8, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x1 {%r9, %r10, %r11, %r12}, {%r8}, {%r7};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r9, %r10, %r11, %r12};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, i32 4, i32 16)
ret <16 x i8> %res
}
-define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to16.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<17>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x2 {%r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r9, %r10, %r11, %r12};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to16.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 16)
ret <32 x i8> %res
}
define <64 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to16.v64i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<19>;
@@ -55,13 +64,13 @@ define <64 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to16.v64i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 16)
ret <64 x i8> %res
}
define <128 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to16.v128i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<36>;
@@ -79,13 +88,13 @@ define <128 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to16.v128i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 16)
ret <128 x i8> %res
}
define <256 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to16.v256i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<71>;
@@ -111,13 +120,13 @@ define <256 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to16.v256i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 16)
ret <256 x i8> %res
}
define <512 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp1to16.v512i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<141>;
@@ -160,32 +169,34 @@ define <512 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <512 x i8> @llvm.nvvm.spdecompress.sp1to16.v512i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 16)
ret <512 x i8> %res
}
; elem_size=16, idx_size=4
-define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to16.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
+; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<11>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to16.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, i32 4, i32 16)
ret <16 x i16> %res
}
define <32 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to16.v32i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<19>;
@@ -199,13 +210,13 @@ define <32 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to16.v32i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4, i32 16)
ret <32 x i16> %res
}
define <64 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to16.v64i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<36>;
@@ -223,13 +234,13 @@ define <64 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to16.v64i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4, i32 16)
ret <64 x i16> %res
}
define <128 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to16.v128i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<70>;
@@ -255,13 +266,13 @@ define <128 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %meta
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to16.v128i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4, i32 16)
ret <128 x i16> %res
}
define <256 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to16.v256i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<139>;
@@ -304,6 +315,6 @@ define <256 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to16.v256i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4, i32 16)
ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
index 6396c02ab58fb6..594c8a25bdaf34 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
@@ -3,26 +3,32 @@
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
-define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to2.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<10>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_1];
-; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x2 {%r3}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x2 {%r9}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
; CHECK-NEXT: ret;
- %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to2.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 2)
ret <4 x i8> %res
}
define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to2.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -33,13 +39,13 @@ define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata,
; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to2.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2, i32 2)
ret <8 x i8> %res
}
define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to2.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -50,13 +56,13 @@ define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to2.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2, i32 2)
ret <16 x i8> %res
}
define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to2.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -68,13 +74,13 @@ define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to2.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2, i32 2)
ret <32 x i8> %res
}
define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to2.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -89,13 +95,13 @@ define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to2.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2, i32 2)
ret <64 x i8> %res
}
define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to2.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -116,30 +122,36 @@ define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to2.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2, i32 2)
ret <128 x i8> %res
}
-define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to2.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<10>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x2 {%r3}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x2 {%r9}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
; CHECK-NEXT: ret;
- %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to2.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 2)
ret <4 x i8> %res
}
define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to2.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -150,13 +162,13 @@ define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata,
; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to2.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 2)
ret <8 x i8> %res
}
define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to2.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -167,13 +179,13 @@ define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to2.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 2)
ret <16 x i8> %res
}
define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to2.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -185,13 +197,13 @@ define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to2.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 2)
ret <32 x i8> %res
}
define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to2.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -206,13 +218,13 @@ define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to2.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 2)
ret <64 x i8> %res
}
define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to2.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -234,30 +246,32 @@ define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to2.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4, i32 2)
ret <128 x i8> %res
}
-define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.sp1to2.v2i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
+; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x1 {%r3}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call <2 x i16> @llvm.nvvm.spdecompress.sp1to2.v2i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, i32 2, i32 2)
ret <2 x i16> %res
}
define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to2.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -268,13 +282,13 @@ define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to2.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2, i32 2)
ret <4 x i16> %res
}
define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to2.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -285,13 +299,13 @@ define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to2.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2, i32 2)
ret <8 x i16> %res
}
define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to2.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -303,13 +317,13 @@ define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to2.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2, i32 2)
ret <16 x i16> %res
}
define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to2.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<26>;
@@ -324,13 +338,13 @@ define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to2.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2, i32 2)
ret <32 x i16> %res
}
define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to2.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<51>;
@@ -351,13 +365,13 @@ define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to2.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2, i32 2)
ret <64 x i16> %res
}
define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to2.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<101>;
@@ -390,30 +404,32 @@ define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to2.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2, i32 2)
ret <128 x i16> %res
}
-define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.sp1to2.v2i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
+; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x1 {%r3}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
- %res = call <2 x i16> @llvm.nvvm.spdecompress.sp1to2.v2i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, i32 4, i32 2)
ret <2 x i16> %res
}
define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to2.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -424,13 +440,13 @@ define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to2.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4, i32 2)
ret <4 x i16> %res
}
define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to2.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -441,13 +457,13 @@ define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to2.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4, i32 2)
ret <8 x i16> %res
}
define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to2.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -459,13 +475,13 @@ define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to2.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4, i32 2)
ret <16 x i16> %res
}
define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to2.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -480,13 +496,13 @@ define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to2.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4, i32 2)
ret <32 x i16> %res
}
define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to2.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -507,13 +523,13 @@ define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to2.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4, i32 2)
ret <64 x i16> %res
}
define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to2.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -547,6 +563,6 @@ define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to2.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4, i32 2)
ret <128 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
index f32a0dffe68bf4..9ff415c9988149 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
@@ -4,43 +4,52 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=2
-define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <1 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-NEXT: .reg .b32 %r<10>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x1 {%r3}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ld.param::func.b8 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: prmt.b32 %r2, %r1, %r3, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r4, %r5, %r6, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r7, %r2, %r4, 0x5410U;
+; CHECK-NEXT: ld.param::func.b32 %r8, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x1 {%r9}, {%r8}, {%r7};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
; CHECK-NEXT: ret;
- %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, i32 2, i32 4)
ret <4 x i8> %res
}
-define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<11>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_1];
-; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x2 {%r9, %r10}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 4)
ret <8 x i8> %res
}
define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to4.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -51,13 +60,13 @@ define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to4.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2, i32 4)
ret <16 x i8> %res
}
define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to4.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -69,13 +78,13 @@ define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to4.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2, i32 4)
ret <32 x i8> %res
}
define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to4.v64i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -89,13 +98,13 @@ define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to4.v64i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2, i32 4)
ret <64 x i8> %res
}
define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to4.v128i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -114,13 +123,13 @@ define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to4.v128i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2, i32 4)
ret <128 x i8> %res
}
define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to4.v256i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -149,48 +158,57 @@ define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to4.v256i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2, i32 4)
ret <256 x i8> %res
}
; elem_size=8, idx_size=4
-define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <1 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-NEXT: .reg .b32 %r<10>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x1 {%r3}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ld.param::func.b8 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: prmt.b32 %r2, %r1, %r3, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r4, %r5, %r6, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r7, %r2, %r4, 0x5410U;
+; CHECK-NEXT: ld.param::func.b32 %r8, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x1 {%r9}, {%r8}, {%r7};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
; CHECK-NEXT: ret;
- %res = call <4 x i8> @llvm.nvvm.spdecompress.sp1to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, i32 4, i32 4)
ret <4 x i8> %res
}
-define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<11>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x2 {%r9, %r10}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 4)
ret <8 x i8> %res
}
define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to4.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -201,13 +219,13 @@ define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to4.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 4)
ret <16 x i8> %res
}
define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to4.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -219,13 +237,13 @@ define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to4.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 4)
ret <32 x i8> %res
}
define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to4.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -239,13 +257,13 @@ define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to4.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 4)
ret <64 x i8> %res
}
define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to4.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -264,13 +282,13 @@ define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to4.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 4)
ret <128 x i8> %res
}
define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to4.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -300,31 +318,33 @@ define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to4.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4, i32 4)
ret <256 x i8> %res
}
; elem_size=16, idx_size=2
-define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
+; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<5>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x1 {%r3, %r4}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, i32 2, i32 4)
ret <4 x i16> %res
}
define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -335,13 +355,13 @@ define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2, i32 4)
ret <8 x i16> %res
}
define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to4.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -353,13 +373,13 @@ define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to4.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2, i32 4)
ret <16 x i16> %res
}
define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to4.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -373,13 +393,13 @@ define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to4.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2, i32 4)
ret <32 x i16> %res
}
define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to4.v64i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<42>;
@@ -398,13 +418,13 @@ define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to4.v64i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2, i32 4)
ret <64 x i16> %res
}
define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to4.v128i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<83>;
@@ -433,13 +453,13 @@ define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to4.v128i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2, i32 4)
ret <128 x i16> %res
}
define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to4.v256i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<165>;
@@ -488,31 +508,33 @@ define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to4.v256i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2)
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2, i32 4)
ret <256 x i16> %res
}
; elem_size=16, idx_size=4
-define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
+; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<5>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x1 {%r3, %r4}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <4 x i16> @llvm.nvvm.spdecompress.sp1to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, i32 4, i32 4)
ret <4 x i16> %res
}
define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -523,13 +545,13 @@ define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to4.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4, i32 4)
ret <8 x i16> %res
}
define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to4.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -541,13 +563,13 @@ define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to4.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4, i32 4)
ret <16 x i16> %res
}
define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to4.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -561,13 +583,13 @@ define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to4.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4, i32 4)
ret <32 x i16> %res
}
define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to4.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -586,13 +608,13 @@ define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to4.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4, i32 4)
ret <64 x i16> %res
}
define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to4.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -621,13 +643,13 @@ define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to4.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4, i32 4)
ret <128 x i16> %res
}
define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to4.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<169>;
@@ -677,6 +699,6 @@ define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to4.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4)
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4, i32 4)
ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
index d55af2e3fcae8e..70e42c4d31df24 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
@@ -4,43 +4,52 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4
-define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <1 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-NEXT: .reg .b32 %r<11>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x1 {%r3, %r4}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ld.param::func.b8 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: prmt.b32 %r2, %r1, %r3, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r4, %r5, %r6, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r7, %r2, %r4, 0x5410U;
+; CHECK-NEXT: ld.param::func.b32 %r8, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x1 {%r9, %r10}, {%r8}, {%r7};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp1to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, i32 4, i32 8)
ret <8 x i8> %res
}
-define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
+define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to8.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<13>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x2 {%r9, %r10, %r11, %r12}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r9, %r10, %r11, %r12};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp1to8.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 8)
ret <16 x i8> %res
}
define <32 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to8.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -52,13 +61,13 @@ define <32 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp1to8.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 8)
ret <32 x i8> %res
}
define <64 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to8.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -72,13 +81,13 @@ define <64 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp1to8.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 8)
ret <64 x i8> %res
}
define <128 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to8.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<39>;
@@ -96,13 +105,13 @@ define <128 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp1to8.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 8)
ret <128 x i8> %res
}
define <256 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to8.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<77>;
@@ -129,13 +138,13 @@ define <256 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp1to8.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 8)
ret <256 x i8> %res
}
define <512 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp1to8.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<153>;
@@ -181,31 +190,33 @@ define <512 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <512 x i8> @llvm.nvvm.spdecompress.sp1to8.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4, i32 8)
ret <512 x i8> %res
}
; elem_size=16, idx_size=4
-define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
+define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to8.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
+; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<7>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x1 {%r3, %r4, %r5, %r6}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp1to8.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, i32 4, i32 8)
ret <8 x i16> %res
}
define <16 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to8.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -217,13 +228,13 @@ define <16 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp1to8.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4, i32 8)
ret <16 x i16> %res
}
define <32 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to8.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -237,13 +248,13 @@ define <32 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp1to8.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4, i32 8)
ret <32 x i16> %res
}
define <64 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to8.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<38>;
@@ -261,13 +272,13 @@ define <64 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp1to8.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4, i32 8)
ret <64 x i16> %res
}
define <128 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to8.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<75>;
@@ -294,13 +305,13 @@ define <128 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp1to8.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4, i32 8)
ret <128 x i16> %res
}
define <256 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to8.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<149>;
@@ -345,6 +356,6 @@ define <256 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <256 x i16> @llvm.nvvm.spdecompress.sp1to8.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4, i32 8)
ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
index 120fe77e1520a1..ef31404cfbccd1 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
@@ -4,27 +4,33 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
-define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<13>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x1 {%r9, %r10, %r11, %r12}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r9, %r10, %r11, %r12};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 16)
ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
define <32 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to16.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -36,14 +42,14 @@ define <32 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to16.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 16)
ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
define <64 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to16.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -57,14 +63,14 @@ define <64 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to16.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 16)
ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
define <128 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to16.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<39>;
@@ -82,14 +88,14 @@ define <128 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to16.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 16)
ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
define <256 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to16.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<77>;
@@ -116,14 +122,14 @@ define <256 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to16.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 16)
ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
define <512 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp2to16.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<153>;
@@ -169,14 +175,14 @@ define <512 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <512 x i8> @llvm.nvvm.spdecompress.sp2to16.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4, i32 16)
ret <512 x i8> %res
}
; elem_size=16, idx_size=4, rpf=1
define <16 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to16.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -188,14 +194,14 @@ define <16 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to16.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4, i32 16)
ret <16 x i16> %res
}
; elem_size=16, idx_size=4, rpf=2
define <32 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to16.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -209,14 +215,14 @@ define <32 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to16.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4, i32 16)
ret <32 x i16> %res
}
; elem_size=16, idx_size=4, rpf=4
define <64 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to16.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<38>;
@@ -234,14 +240,14 @@ define <64 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metad
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to16.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4, i32 16)
ret <64 x i16> %res
}
; elem_size=16, idx_size=4, rpf=8
define <128 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to16.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<75>;
@@ -268,14 +274,14 @@ define <128 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to16.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4, i32 16)
ret <128 x i16> %res
}
; elem_size=16, idx_size=4, rpf=16
define <256 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to16.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<149>;
@@ -320,6 +326,6 @@ define <256 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to16.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4, i32 16)
ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
index 88ad05acd0945e..8370fe02adf01f 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
@@ -4,27 +4,33 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=2, rpf=1
-define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<10>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x1 {%r3}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x1 {%r9}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
; CHECK-NEXT: ret;
- %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 4)
ret <4 x i8> %res
}
; elem_size=8, idx_size=2, rpf=2
define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -35,14 +41,14 @@ define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata,
; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2, i32 4)
ret <8 x i8> %res
}
; elem_size=8, idx_size=2, rpf=4
define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -53,14 +59,14 @@ define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 2, i32 4)
ret <16 x i8> %res
}
; elem_size=8, idx_size=2, rpf=8
define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to4.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -72,14 +78,14 @@ define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to4.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, i32 2, i32 4)
ret <32 x i8> %res
}
; elem_size=8, idx_size=2, rpf=16
define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to4.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -94,14 +100,14 @@ define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to4.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, i32 2, i32 4)
ret <64 x i8> %res
}
; elem_size=8, idx_size=2, rpf=32
define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to4.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -122,14 +128,14 @@ define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to4.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, i32 2, i32 4)
ret <128 x i8> %res
}
; elem_size=8, idx_size=2, rpf=64
define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to4.v256i8.v8i32.v128i8(<8 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v128i8(<8 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -163,32 +169,38 @@ define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to4.v256i8.v8i32.v128i8(<8 x i32> %metadata, <128 x i8> %compdata, i32 2)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v128i8(<8 x i32> %metadata, <128 x i8> %compdata, i32 2, i32 4)
ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=1
-define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<10>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x1 {%r3}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x1 {%r9}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
; CHECK-NEXT: ret;
- %res = call <4 x i8> @llvm.nvvm.spdecompress.sp2to4.v4i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 4)
ret <4 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -199,14 +211,14 @@ define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata,
; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to4.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 4)
ret <8 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -217,14 +229,14 @@ define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to4.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 4)
ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to4.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -236,14 +248,14 @@ define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %me
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to4.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 4)
ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to4.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -258,14 +270,14 @@ define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to4.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 4)
ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to4.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -287,14 +299,14 @@ define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to4.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4, i32 4)
ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=64
define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to4.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<113>;
@@ -330,14 +342,14 @@ define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to4.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4, i32 4)
ret <256 x i8> %res
}
; elem_size=16, idx_size=2, rpf=1
define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -348,14 +360,14 @@ define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2)
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 2, i32 4)
ret <4 x i16> %res
}
; elem_size=16, idx_size=2, rpf=2
define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -366,14 +378,14 @@ define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 2, i32 4)
ret <8 x i16> %res
}
; elem_size=16, idx_size=2, rpf=4
define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to4.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -385,14 +397,14 @@ define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to4.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 2, i32 4)
ret <16 x i16> %res
}
; elem_size=16, idx_size=2, rpf=8
define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to4.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<26>;
@@ -407,14 +419,14 @@ define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to4.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, i32 2, i32 4)
ret <32 x i16> %res
}
; elem_size=16, idx_size=2, rpf=16
define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to4.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<51>;
@@ -435,14 +447,14 @@ define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to4.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, i32 2, i32 4)
ret <64 x i16> %res
}
; elem_size=16, idx_size=2, rpf=32
define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to4.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<101>;
@@ -475,14 +487,14 @@ define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to4.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, i32 2, i32 4)
ret <128 x i16> %res
}
; elem_size=16, idx_size=2, rpf=64
define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32> %metadata, <128 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to4.v256i16.v8i32.v128i16(<8 x i32> %metadata, <128 x i16> %compdata, /* idx_size=.b2 */ i32 2)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v128i16(<8 x i32> %metadata, <128 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<201>;
@@ -540,14 +552,14 @@ define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r77, %r78, %r79, %r80};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r73, %r74, %r75, %r76};
; CHECK-NEXT: ret;
- %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to4.v256i16.v8i32.v128i16(<8 x i32> %metadata, <128 x i16> %compdata, i32 2)
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v128i16(<8 x i32> %metadata, <128 x i16> %compdata, i32 2, i32 4)
ret <256 x i16> %res
}
; elem_size=16, idx_size=4, rpf=1
define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -558,14 +570,14 @@ define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <4 x i16> @llvm.nvvm.spdecompress.sp2to4.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4, i32 4)
ret <4 x i16> %res
}
; elem_size=16, idx_size=4, rpf=2
define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -576,14 +588,14 @@ define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to4.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4, i32 4)
ret <8 x i16> %res
}
; elem_size=16, idx_size=4, rpf=4
define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to4.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -595,14 +607,14 @@ define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to4.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4, i32 4)
ret <16 x i16> %res
}
; elem_size=16, idx_size=4, rpf=8
define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to4.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -617,14 +629,14 @@ define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to4.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4, i32 4)
ret <32 x i16> %res
}
; elem_size=16, idx_size=4, rpf=16
define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to4.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -645,14 +657,14 @@ define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to4.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4, i32 4)
ret <64 x i16> %res
}
; elem_size=16, idx_size=4, rpf=32
define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to4.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -686,14 +698,14 @@ define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to4.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4, i32 4)
ret <128 x i16> %res
}
; elem_size=16, idx_size=4, rpf=64
define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to4.v256i16.v16i32.v128i16(<16 x i32> %metadata, <128 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v16i32.v128i16(<16 x i32> %metadata, <128 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<209>;
@@ -753,6 +765,6 @@ define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r85, %r86, %r87, %r88};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r81, %r82, %r83, %r84};
; CHECK-NEXT: ret;
- %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to4.v256i16.v16i32.v128i16(<16 x i32> %metadata, <128 x i16> %compdata, i32 4)
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v16i32.v128i16(<16 x i32> %metadata, <128 x i16> %compdata, i32 4, i32 4)
ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
index f27d04cadec7f1..9a58dee3ff4e75 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
@@ -4,27 +4,33 @@
; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
-define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
+define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-NEXT: .reg .b16 %rs<3>;
+; CHECK-NEXT: .reg .b32 %r<11>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x1 {%r3, %r4}, {%r1}, {%r2};
-; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
+; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
+; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
+; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x1 {%r9, %r10}, {%r1}, {%r8};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r9, %r10};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp2to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 8)
ret <8 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
define <16 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to8.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -35,14 +41,14 @@ define <16 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp2to8.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 8)
ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
define <32 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to8.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -54,14 +60,14 @@ define <32 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp2to8.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 8)
ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
define <64 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to8.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -75,14 +81,14 @@ define <64 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %me
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp2to8.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 8)
ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
define <128 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to8.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -101,14 +107,14 @@ define <128 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp2to8.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 8)
ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
define <256 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to8.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -138,14 +144,14 @@ define <256 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp2to8.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4, i32 8)
ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=64
define <512 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp2to8.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<177>;
@@ -197,14 +203,14 @@ define <512 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <512 x i8> @llvm.nvvm.spdecompress.sp2to8.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4)
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4, i32 8)
ret <512 x i8> %res
}
; elem_size=16, idx_size=4, rpf=1
define <8 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to8.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -215,14 +221,14 @@ define <8 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <8 x i16> @llvm.nvvm.spdecompress.sp2to8.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4)
+ %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, i32 4, i32 8)
ret <8 x i16> %res
}
; elem_size=16, idx_size=4, rpf=2
define <16 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to8.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -234,14 +240,14 @@ define <16 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i16> @llvm.nvvm.spdecompress.sp2to8.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4)
+ %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, i32 4, i32 8)
ret <16 x i16> %res
}
; elem_size=16, idx_size=4, rpf=4
define <32 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to8.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -255,14 +261,14 @@ define <32 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
; CHECK-NEXT: ret;
- %res = call <32 x i16> @llvm.nvvm.spdecompress.sp2to8.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4)
+ %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, i32 4, i32 8)
ret <32 x i16> %res
}
; elem_size=16, idx_size=4, rpf=8
define <64 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to8.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -281,14 +287,14 @@ define <64 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: ret;
- %res = call <64 x i16> @llvm.nvvm.spdecompress.sp2to8.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4)
+ %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, i32 4, i32 8)
ret <64 x i16> %res
}
; elem_size=16, idx_size=4, rpf=16
define <128 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to8.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -317,14 +323,14 @@ define <128 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
; CHECK-NEXT: ret;
- %res = call <128 x i16> @llvm.nvvm.spdecompress.sp2to8.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4)
+ %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, i32 4, i32 8)
ret <128 x i16> %res
}
; elem_size=16, idx_size=4, rpf=32
define <256 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to8.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<169>;
@@ -374,6 +380,6 @@ define <256 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
; CHECK-NEXT: ret;
- %res = call <256 x i16> @llvm.nvvm.spdecompress.sp2to8.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4)
+ %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, i32 4, i32 8)
ret <256 x i16> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
index dae7437d62cafa..d1269a0cf064d1 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp4to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -17,14 +17,14 @@ define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp4to16.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 16)
ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
define <32 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp4to16.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -36,14 +36,14 @@ define <32 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp4to16.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 16)
ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
define <64 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp4to16.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -57,14 +57,14 @@ define <64 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %m
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp4to16.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 16)
ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
define <128 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp4to16.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -83,14 +83,14 @@ define <128 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp4to16.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 16)
ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
define <256 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp4to16.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -120,14 +120,14 @@ define <256 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp4to16.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4, i32 16)
ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
define <512 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp4to16.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<177>;
@@ -179,6 +179,6 @@ define <512 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <512 x i8> @llvm.nvvm.spdecompress.sp4to16.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4)
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4, i32 16)
ret <512 x i8> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
index 1cf8af76cce4b0..30fba70f9989cc 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.sp4to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -17,14 +17,14 @@ define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x1 {%r3, %r4}, {%r1}, {%r2};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
- %res = call <8 x i8> @llvm.nvvm.spdecompress.sp4to8.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4)
+ %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 4, i32 8)
ret <8 x i8> %res
}
; elem_size=8, idx_size=4, rpf=2
define <16 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.sp4to8.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -35,14 +35,14 @@ define <16 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
; CHECK-NEXT: ret;
- %res = call <16 x i8> @llvm.nvvm.spdecompress.sp4to8.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4)
+ %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 8)
ret <16 x i8> %res
}
; elem_size=8, idx_size=4, rpf=4
define <32 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.sp4to8.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -54,14 +54,14 @@ define <32 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %me
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
; CHECK-NEXT: ret;
- %res = call <32 x i8> @llvm.nvvm.spdecompress.sp4to8.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4)
+ %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, i32 4, i32 8)
ret <32 x i8> %res
}
; elem_size=8, idx_size=4, rpf=8
define <64 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.sp4to8.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -76,14 +76,14 @@ define <64 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %me
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
; CHECK-NEXT: ret;
- %res = call <64 x i8> @llvm.nvvm.spdecompress.sp4to8.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4)
+ %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, i32 4, i32 8)
ret <64 x i8> %res
}
; elem_size=8, idx_size=4, rpf=16
define <128 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.sp4to8.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -105,14 +105,14 @@ define <128 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
; CHECK-NEXT: ret;
- %res = call <128 x i8> @llvm.nvvm.spdecompress.sp4to8.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4)
+ %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, i32 4, i32 8)
ret <128 x i8> %res
}
; elem_size=8, idx_size=4, rpf=32
define <256 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.sp4to8.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<113>;
@@ -148,14 +148,14 @@ define <256 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
; CHECK-NEXT: ret;
- %res = call <256 x i8> @llvm.nvvm.spdecompress.sp4to8.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4)
+ %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, i32 4, i32 8)
ret <256 x i8> %res
}
; elem_size=8, idx_size=4, rpf=64
define <512 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32> %metadata, <256 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.sp4to8.v512i8.v32i32.v256i8(<32 x i32> %metadata, <256 x i8> %compdata, /* idx_size=.b4 */ i32 4)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v32i32.v256i8(<32 x i32> %metadata, <256 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<225>;
@@ -219,6 +219,6 @@ define <512 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32>
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r101, %r102, %r103, %r104};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r97, %r98, %r99, %r100};
; CHECK-NEXT: ret;
- %res = call <512 x i8> @llvm.nvvm.spdecompress.sp4to8.v512i8.v32i32.v256i8(<32 x i32> %metadata, <256 x i8> %compdata, i32 4)
+ %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v32i32.v256i8(<32 x i32> %metadata, <256 x i8> %compdata, i32 4, i32 8)
ret <512 x i8> %res
}
>From 46e82b87050876ac15ca9e96b965d7a6d4bd9947 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Tue, 15 Sep 2026 17:47:28 +0200
Subject: [PATCH 05/10] [NVVM][NVPTX] Verify and simplify sp{decompress}
intrinsic lowering
---
llvm/include/llvm/IR/IntrinsicsNVVM.td | 7 +-
llvm/include/llvm/IR/NVVMIntrinsicUtils.h | 86 ++----
llvm/lib/IR/CMakeLists.txt | 1 +
llvm/lib/IR/NVVMIntrinsicUtils.cpp | 6 +-
llvm/lib/IR/Verifier.cpp | 1 +
llvm/lib/IR/VerifierInternal.h | 7 +
llvm/lib/IR/VerifierNVPTX.cpp | 133 +++++++++
.../NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp | 18 +-
llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp | 258 ++++--------------
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp | 155 ++++++-----
.../Target/NVPTX/NVPTXSelectionDAGInfo.cpp | 6 +
llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h | 2 +
llvm/test/CodeGen/NVPTX/spcompress-invalid.ll | 34 ++-
llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll | 56 ++--
.../CodeGen/NVPTX/spdecompress-invalid.ll | 32 +--
.../CodeGen/NVPTX/spdecompress-sp1to16.ll | 22 +-
.../test/CodeGen/NVPTX/spdecompress-sp1to2.ll | 52 ++--
.../test/CodeGen/NVPTX/spdecompress-sp1to4.ll | 56 ++--
.../test/CodeGen/NVPTX/spdecompress-sp1to8.ll | 26 +-
.../CodeGen/NVPTX/spdecompress-sp2to16.ll | 22 +-
.../test/CodeGen/NVPTX/spdecompress-sp2to4.ll | 56 ++--
.../test/CodeGen/NVPTX/spdecompress-sp2to8.ll | 26 +-
.../CodeGen/NVPTX/spdecompress-sp4to16.ll | 12 +-
.../test/CodeGen/NVPTX/spdecompress-sp4to8.ll | 14 +-
llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn | 1 +
25 files changed, 521 insertions(+), 568 deletions(-)
create mode 100644 llvm/lib/IR/VerifierNVPTX.cpp
diff --git a/llvm/include/llvm/IR/IntrinsicsNVVM.td b/llvm/include/llvm/IR/IntrinsicsNVVM.td
index 380341b352d9b2..b9c493c73d4d6b 100644
--- a/llvm/include/llvm/IR/IntrinsicsNVVM.td
+++ b/llvm/include/llvm/IR/IntrinsicsNVVM.td
@@ -1338,8 +1338,7 @@ class NVVMPureIntrinsic<list<LLVMType> ret_types,
class NVVM_SP_IDX_ARGPROPS<ArgIndex IdxSizeIdx> {
list<IntrinsicProperty> Prop = [
RangeList<IdxSizeIdx, [2, 4]>,
- ArgInfo<IdxSizeIdx, [ArgName<"idx_size">,
- ImmArgPrinter<"printSPIdxSize">]>
+ ArgInfo<IdxSizeIdx, [ArgName<"idx_size">]>
];
}
@@ -4232,7 +4231,7 @@ def int_nvvm_tensormap_replace_fill_mode :
class SPCompress :
DefaultAttrsIntrinsicFlags<
- [llvm_anyint_ty, // mdata
+ [llvm_anyint_ty, // mdata
llvm_any_vector_int_ty], // cdata
[llvm_any_vector_int_ty, // data
llvm_i32_ty], // spdesc
@@ -4252,7 +4251,7 @@ def int_nvvm_spcompress : SPCompress;
class SPDecompress :
DefaultAttrsIntrinsicFlags<
[llvm_any_vector_int_ty], // data
- [llvm_anyint_ty, // mdata
+ [llvm_anyint_ty, // mdata
llvm_any_vector_int_ty], // cdata
[llvm_i32_ty, // idx_size
llvm_i32_ty], // num_tgt
diff --git a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
index 17ca42b3ea99a5..65ba00711697e9 100644
--- a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
+++ b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
@@ -15,15 +15,16 @@
#ifndef LLVM_IR_NVVMINTRINSICUTILS_H
#define LLVM_IR_NVVMINTRINSICUTILS_H
+#include <optional>
#include <stdint.h>
#include "llvm/ADT/APFloat.h"
#include "llvm/ADT/APInt.h"
-#include "llvm/ADT/Enum.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/IR/Constants.h"
#include "llvm/IR/Intrinsics.h"
#include "llvm/IR/IntrinsicsNVPTX.h"
+#include "llvm/Support/MathExtras.h"
#include "llvm/Support/raw_ostream.h"
namespace llvm {
@@ -216,26 +217,8 @@ enum class MBarrierLayout : uint8_t {
V1 = 1,
};
-enum class SPElemSize : uint64_t {
- B8 = 8,
- B16 = 16,
-};
-
-enum class SPIdxSize : uint64_t {
- B2 = 2,
- B4 = 4,
-};
-
-enum class SPRepeatFactor : uint64_t {
- X1 = 1,
- X2 = 2,
- X4 = 4,
- X8 = 8,
- X16 = 16,
- X32 = 32,
- X64 = 64,
-};
-
+// Sparse intrinsic layouts are shared by the IR verifier and the NVPTX
+// instruction printer.
struct SPOperandLayout {
unsigned MetadataSize;
unsigned CompressedDataSize;
@@ -246,40 +229,16 @@ struct SPOperandLayout {
// of spcompress and spdecompress to 253 32-bit registers.
inline constexpr unsigned MaxSPOperandRegisters = 253;
-inline StringRef getSPElemSizeName(SPElemSize ElemSize) {
- static constexpr EnumStringDef<SPElemSize> SPElemSizeNameDefs[] = {
- {{".b8"}, SPElemSize::B8},
- {{".b16"}, SPElemSize::B16},
- };
- static constexpr auto SPElemSizeNames =
- BUILD_ENUM_STRINGS(SPElemSizeNameDefs);
- return EnumStrings(SPElemSizeNames).toString(ElemSize);
-}
-
-inline StringRef getSPIdxSizeName(SPIdxSize IdxSize) {
- static constexpr EnumStringDef<SPIdxSize> SPIdxSizeNameDefs[] = {
- {{".b2"}, SPIdxSize::B2},
- {{".b4"}, SPIdxSize::B4},
- };
- static constexpr auto SPIdxSizeNames = BUILD_ENUM_STRINGS(SPIdxSizeNameDefs);
- return EnumStrings(SPIdxSizeNames).toString(IdxSize);
+inline bool isValidSPElemSize(unsigned ElemSize) {
+ return ElemSize == 8 || ElemSize == 16;
}
-inline StringRef getSPRepeatFactorName(SPRepeatFactor RepeatFactor) {
- static constexpr EnumStringDef<SPRepeatFactor> SPRepeatFactorNameDefs[] = {
- {{".x1"}, SPRepeatFactor::X1}, {{".x2"}, SPRepeatFactor::X2},
- {{".x4"}, SPRepeatFactor::X4}, {{".x8"}, SPRepeatFactor::X8},
- {{".x16"}, SPRepeatFactor::X16}, {{".x32"}, SPRepeatFactor::X32},
- {{".x64"}, SPRepeatFactor::X64},
- };
- static constexpr auto SPRepeatFactorNames =
- BUILD_ENUM_STRINGS(SPRepeatFactorNameDefs);
- return EnumStrings(SPRepeatFactorNames).toString(RepeatFactor);
+inline bool isValidSPIdxSize(unsigned IdxSize) {
+ return IdxSize == 2 || IdxSize == 4;
}
inline bool isValidSPRepeatFactor(unsigned RepeatFactor) {
- return !getSPRepeatFactorName(static_cast<SPRepeatFactor>(RepeatFactor))
- .empty();
+ return isPowerOf2_32(RepeatFactor) && RepeatFactor <= 64;
}
inline bool isValidSPDecompressFactor(unsigned NumSrc, unsigned NumTgt) {
@@ -298,16 +257,12 @@ inline bool isValidSPDecompressFactor(unsigned NumSrc, unsigned NumTgt) {
inline std::optional<SPOperandLayout>
getSPCompressLayout(unsigned ElemSize, unsigned IdxSize,
unsigned RepeatFactor) {
- if (getSPElemSizeName(static_cast<SPElemSize>(ElemSize)).empty() ||
- getSPIdxSizeName(static_cast<SPIdxSize>(IdxSize)).empty() ||
+ if (!isValidSPElemSize(ElemSize) || !isValidSPIdxSize(IdxSize) ||
!isValidSPRepeatFactor(RepeatFactor))
return std::nullopt;
- SPOperandLayout Layout = {
- divideCeil(RepeatFactor * IdxSize, ElemSize),
- RepeatFactor,
- RepeatFactor * 2,
- };
+ SPOperandLayout Layout = {divideCeil(RepeatFactor * IdxSize, ElemSize),
+ RepeatFactor, RepeatFactor * 2};
if (Layout.MetadataSize + Layout.CompressedDataSize + Layout.DataSize >
MaxSPOperandRegisters)
return std::nullopt;
@@ -318,22 +273,18 @@ inline std::optional<SPOperandLayout>
getSPDecompressLayout(unsigned NumSrc, unsigned NumTgt, unsigned ElemSize,
unsigned IdxSize, unsigned RepeatFactor) {
if (!isValidSPDecompressFactor(NumSrc, NumTgt) ||
- getSPElemSizeName(static_cast<SPElemSize>(ElemSize)).empty() ||
- getSPIdxSizeName(static_cast<SPIdxSize>(IdxSize)).empty() ||
- !isValidSPRepeatFactor(RepeatFactor))
- return std::nullopt;
- if (NumSrc * ElemSize > 32 || (IdxSize == 2 && NumTgt > 4))
+ !isValidSPElemSize(ElemSize) || !isValidSPIdxSize(IdxSize) ||
+ !isValidSPRepeatFactor(RepeatFactor) || NumSrc * ElemSize > 32 ||
+ (IdxSize == 2 && NumTgt > 4))
return std::nullopt;
unsigned DataBits = NumTgt * ElemSize * RepeatFactor;
if (DataBits < 32 || DataBits > 4096)
return std::nullopt;
- SPOperandLayout Layout = {
- divideCeil(NumSrc * IdxSize * RepeatFactor, 32),
- divideCeil(NumSrc * ElemSize * RepeatFactor, 32),
- divideCeil(DataBits, 32),
- };
+ SPOperandLayout Layout = {divideCeil(NumSrc * IdxSize * RepeatFactor, 32),
+ divideCeil(NumSrc * ElemSize * RepeatFactor, 32),
+ divideCeil(DataBits, 32)};
if (Layout.MetadataSize + Layout.CompressedDataSize + Layout.DataSize >
MaxSPOperandRegisters)
return std::nullopt;
@@ -359,7 +310,6 @@ LLVM_ABI void printTcgen05CollectorUsageOp(raw_ostream &OS,
LLVM_ABI void printTcgen05MMACollectorBBuffer(raw_ostream &OS,
const Constant *ImmArgVal);
-LLVM_ABI void printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal);
LLVM_ABI void printTensormapElemType(raw_ostream &OS,
const Constant *ImmArgVal);
LLVM_ABI void printTensormapInterleaveLayout(raw_ostream &OS,
diff --git a/llvm/lib/IR/CMakeLists.txt b/llvm/lib/IR/CMakeLists.txt
index 3037f01083308b..24dc690ae97594 100644
--- a/llvm/lib/IR/CMakeLists.txt
+++ b/llvm/lib/IR/CMakeLists.txt
@@ -80,6 +80,7 @@ add_llvm_component_library(LLVMCore
VectorTypeUtils.cpp
Verifier.cpp
VerifierAMDGPU.cpp
+ VerifierNVPTX.cpp
VFABIDemangler.cpp
RuntimeLibcalls.cpp
diff --git a/llvm/lib/IR/NVVMIntrinsicUtils.cpp b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
index 933ab39104cb14..e5a45fd5e9cd9b 100644
--- a/llvm/lib/IR/NVVMIntrinsicUtils.cpp
+++ b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
@@ -11,6 +11,7 @@
//===----------------------------------------------------------------------===//
#include "llvm/IR/NVVMIntrinsicUtils.h"
+#include "llvm/ADT/StringRef.h"
using namespace llvm;
using namespace nvvm;
@@ -136,11 +137,6 @@ void nvvm::printTcgen05MMACollectorBBuffer(raw_ostream &OS,
}
}
-void nvvm::printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal) {
- if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
- OS << getSPIdxSizeName(static_cast<SPIdxSize>(CI->getZExtValue()));
-}
-
void nvvm::printTensormapElemType(raw_ostream &OS, const Constant *ImmArgVal) {
static constexpr StringRef TensormapElemTypes[] = {
"u8", "u16", "u32", "s32", "u64", "s64",
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 1440b95896474f..25f1f8b6437659 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -7415,6 +7415,7 @@ void Verifier::visitIntrinsicCall(Intrinsic::ID ID, CallBase &Call) {
// Target-specific intrinsic call checks.
verifyAMDGPUIntrinsicCall(*this, ID, Call);
+ verifyNVPTXIntrinsicCall(*this, ID, Call);
}
/// Carefully grab the subprogram from a local scope.
diff --git a/llvm/lib/IR/VerifierInternal.h b/llvm/lib/IR/VerifierInternal.h
index 0a7381f27d4045..f38947de2f0bb5 100644
--- a/llvm/lib/IR/VerifierInternal.h
+++ b/llvm/lib/IR/VerifierInternal.h
@@ -230,6 +230,13 @@ bool isAMDGPUCallBrIntrinsic(Intrinsic::ID ID);
//==============================================================================
+// NVPTX-specific verification functions
+
+void verifyNVPTXIntrinsicCall(VerifierSupport &VS, Intrinsic::ID ID,
+ CallBase &Call);
+
+//==============================================================================
+
} // namespace llvm
#endif // LLVM_LIB_IR_VERIFIERINTERNAL_H
diff --git a/llvm/lib/IR/VerifierNVPTX.cpp b/llvm/lib/IR/VerifierNVPTX.cpp
new file mode 100644
index 00000000000000..e0e1ec292be709
--- /dev/null
+++ b/llvm/lib/IR/VerifierNVPTX.cpp
@@ -0,0 +1,133 @@
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// This file contains NVPTX-specific IR verification logic. These checks are
+/// always compiled and linked as part of LLVMCore.
+///
+//===----------------------------------------------------------------------===//
+
+#include "VerifierInternal.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/DerivedTypes.h"
+#include "llvm/IR/IntrinsicsNVPTX.h"
+#include "llvm/IR/NVVMIntrinsicUtils.h"
+#include "llvm/Support/MathExtras.h"
+#include <optional>
+
+using namespace llvm;
+
+#define Check(C, ...) \
+ do { \
+ if (!(C)) { \
+ VS.CheckFailed(__VA_ARGS__); \
+ return; \
+ } \
+ } while (false)
+
+namespace {
+
+struct SPVectorInfo {
+ unsigned ElemSize;
+ unsigned NumElements;
+ unsigned NumRegisters;
+};
+
+} // namespace
+
+static std::optional<SPVectorInfo> getSPVectorInfo(Type *Ty) {
+ auto *VT = dyn_cast<FixedVectorType>(Ty);
+ if (!VT)
+ return std::nullopt;
+ auto *ElemTy = dyn_cast<IntegerType>(VT->getElementType());
+ if (!ElemTy)
+ return std::nullopt;
+ unsigned ElemSize = ElemTy->getBitWidth();
+ if (ElemSize != 8 && ElemSize != 16)
+ return std::nullopt;
+ unsigned NumElements = VT->getNumElements();
+ return SPVectorInfo{ElemSize, NumElements,
+ divideCeil(NumElements, 32 / ElemSize)};
+}
+
+static std::optional<unsigned> getSPMetadataRegisters(Type *Ty) {
+ if (Ty->isIntegerTy(32))
+ return 1;
+ auto *VT = dyn_cast<FixedVectorType>(Ty);
+ if (!VT || !VT->getElementType()->isIntegerTy(32))
+ return std::nullopt;
+ return VT->getNumElements();
+}
+
+static void verifySPCompress(VerifierSupport &VS, CallBase &Call) {
+ auto *ResultTy = dyn_cast<StructType>(Call.getType());
+ Check(ResultTy && ResultTy->getNumElements() == 2,
+ "invalid llvm.nvvm.spcompress result type", &Call);
+
+ auto MDataRegs = getSPMetadataRegisters(ResultTy->getElementType(0));
+ auto CData = getSPVectorInfo(ResultTy->getElementType(1));
+ auto Data = getSPVectorInfo(Call.getArgOperand(0)->getType());
+ Check(MDataRegs && CData && Data && CData->ElemSize == Data->ElemSize,
+ "invalid llvm.nvvm.spcompress operand or result type", &Call);
+
+ unsigned IdxSize = cast<ConstantInt>(Call.getArgOperand(2))->getZExtValue();
+ unsigned NumTgt = cast<ConstantInt>(Call.getArgOperand(3))->getZExtValue();
+ Check(NumTgt == 4 && Data->NumElements % NumTgt == 0 &&
+ CData->NumElements == 2 * (Data->NumElements / NumTgt) &&
+ Data->NumRegisters % 2 == 0,
+ "invalid llvm.nvvm.spcompress layout", &Call);
+
+ unsigned RepeatFactor = Data->NumRegisters / 2;
+ auto Layout =
+ nvvm::getSPCompressLayout(Data->ElemSize, IdxSize, RepeatFactor);
+ Check(Layout && *MDataRegs == Layout->MetadataSize &&
+ CData->NumRegisters == Layout->CompressedDataSize &&
+ Data->NumRegisters == Layout->DataSize,
+ "invalid llvm.nvvm.spcompress layout", &Call);
+}
+
+static void verifySPDecompress(VerifierSupport &VS, CallBase &Call) {
+ auto Data = getSPVectorInfo(Call.getType());
+ auto MDataRegs = getSPMetadataRegisters(Call.getArgOperand(0)->getType());
+ auto CData = getSPVectorInfo(Call.getArgOperand(1)->getType());
+ Check(Data && MDataRegs && CData && Data->ElemSize == CData->ElemSize,
+ "invalid llvm.nvvm.spdecompress operand or result type", &Call);
+
+ unsigned IdxSize = cast<ConstantInt>(Call.getArgOperand(2))->getZExtValue();
+ unsigned NumTgt = cast<ConstantInt>(Call.getArgOperand(3))->getZExtValue();
+ Check(NumTgt != 0 && Data->NumElements % NumTgt == 0,
+ "invalid llvm.nvvm.spdecompress layout", &Call);
+
+ unsigned RepeatFactor = Data->NumElements / NumTgt;
+ Check(RepeatFactor != 0 && CData->NumElements % RepeatFactor == 0,
+ "invalid llvm.nvvm.spdecompress layout", &Call);
+
+ unsigned NumSrc = CData->NumElements / RepeatFactor;
+ auto Layout = nvvm::getSPDecompressLayout(NumSrc, NumTgt, Data->ElemSize,
+ IdxSize, RepeatFactor);
+ Check(Layout && *MDataRegs == Layout->MetadataSize &&
+ CData->NumRegisters == Layout->CompressedDataSize &&
+ Data->NumRegisters == Layout->DataSize,
+ "invalid llvm.nvvm.spdecompress layout", &Call);
+}
+
+void llvm::verifyNVPTXIntrinsicCall(VerifierSupport &VS, Intrinsic::ID ID,
+ CallBase &Call) {
+ switch (ID) {
+ default:
+ return;
+ case Intrinsic::nvvm_spcompress:
+ verifySPCompress(VS, Call);
+ return;
+ case Intrinsic::nvvm_spdecompress:
+ verifySPDecompress(VS, Call);
+ return;
+ }
+}
+
+#undef Check
diff --git a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
index 42dc737ce7e463..5274f1cacad4d6 100644
--- a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
+++ b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
@@ -496,26 +496,20 @@ void NVPTXInstPrinter::printSPQualifier(const MCInst *MI, int OpNum,
unsigned Imm = MI->getOperand(OpNum).getImm();
if (Modifier == "elem_size") {
- StringRef ElemSize =
- nvvm::getSPElemSizeName(static_cast<nvvm::SPElemSize>(Imm));
- assert(!ElemSize.empty() && "invalid SP elem_size");
- O << ElemSize;
+ assert(nvvm::isValidSPElemSize(Imm) && "invalid SP elem_size");
+ O << ".b" << Imm;
return;
}
if (Modifier == "idx_size") {
- StringRef IdxSize =
- nvvm::getSPIdxSizeName(static_cast<nvvm::SPIdxSize>(Imm));
- assert(!IdxSize.empty() && "invalid SP idx_size");
- O << IdxSize;
+ assert(nvvm::isValidSPIdxSize(Imm) && "invalid SP idx_size");
+ O << ".b" << Imm;
return;
}
if (Modifier == "repeat_factor") {
- StringRef RepeatFactor =
- nvvm::getSPRepeatFactorName(static_cast<nvvm::SPRepeatFactor>(Imm));
- assert(!RepeatFactor.empty() && "invalid SP repeat_factor");
- O << RepeatFactor;
+ assert(nvvm::isValidSPRepeatFactor(Imm) && "invalid SP repeat_factor");
+ O << ".x" << Imm;
return;
}
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
index e085da8f0c5e7d..68d262bf125329 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
@@ -115,7 +115,6 @@ class NVPTXDAGToDAGISel : public SelectionDAGISel {
#include "NVPTXGenDAGISel.inc"
void Select(SDNode *N) override;
- bool tryIntrinsicNoChain(SDNode *N);
bool tryIntrinsicChain(SDNode *N);
bool tryIntrinsicVoid(SDNode *N);
void SelectTexSurfHandle(SDNode *N);
@@ -139,8 +138,7 @@ class NVPTXDAGToDAGISel : public SelectionDAGISel {
bool IsIm2Col = false);
void SelectTcgen05Ld(SDNode *N, bool hasOffset = false);
void SelectTcgen05St(SDNode *N, bool hasOffset = false);
- void selectSPCompress(SDNode *N);
- void selectSPDecompress(SDNode *N);
+ void selectSPNode(SDNode *N);
void selectAtomicSwap128(SDNode *N);
inline SDValue getI32Imm(unsigned Imm, const SDLoc &DL) {
@@ -292,10 +290,10 @@ void NVPTXDAGToDAGISel::Select(SDNode *N) {
if (tryStoreVector(N))
return;
break;
- case ISD::INTRINSIC_WO_CHAIN:
- if (tryIntrinsicNoChain(N))
- return;
- break;
+ case NVPTXISD::SPCOMPRESS:
+ case NVPTXISD::SPDECOMPRESS:
+ selectSPNode(N);
+ return;
case ISD::INTRINSIC_W_CHAIN:
if (tryIntrinsicChain(N))
return;
@@ -448,218 +446,66 @@ void NVPTXDAGToDAGISel::SelectTcgen05Ld(SDNode *N, bool hasOffset) {
}
}
-static std::optional<unsigned> getSPDataElemSize(EVT VT) {
- if (VT != MVT::v4i8 && VT != MVT::v2i16)
- return std::nullopt;
- return static_cast<unsigned>(VT.getScalarSizeInBits());
-}
-
-// Keep these lists in sync with their counterparts in NVPTXIntrinsics.td. A
-// C++-only count references a nonexistent opcode and fails to compile; a
-// TableGen-only count reaches llvm_unreachable in the selectors below.
-#define SPCOMPRESS_OPCODE_CASES(CASE_MACRO) \
- /* .b8.b2 and .b16.b4: ceil(repeat_factor / 4) + repeat_factor. */ \
- CASE_MACRO(2) /* .x1 */ \
- CASE_MACRO(3) /* .x2 */ \
- CASE_MACRO(5) /* .x4 */ \
- CASE_MACRO(10) /* .x8 */ \
- CASE_MACRO(20) /* .x16 */ \
- CASE_MACRO(40) /* .x32 */ \
- CASE_MACRO(80) /* .x64 */ \
- /* .b16.b2: additional unique counts after the duplicate small cases. */ \
- CASE_MACRO(9) /* .x8 */ \
- CASE_MACRO(18) /* .x16 */ \
- CASE_MACRO(36) /* .x32 */ \
- CASE_MACRO(72) /* .x64 */ \
- /* .b8.b4: additional unique counts after the duplicate small cases. */ \
- CASE_MACRO(6) /* .x4 */ \
- CASE_MACRO(12) /* .x8 */ \
- CASE_MACRO(24) /* .x16 */ \
- CASE_MACRO(48) /* .x32 */ \
- CASE_MACRO(96) /* .x64 */
-
-#define SPDECOMPRESS_OPCODE_CASES(CASE_MACRO) \
- CASE_MACRO(1) \
- CASE_MACRO(2) \
- CASE_MACRO(4) \
- CASE_MACRO(8) \
- CASE_MACRO(16) \
- CASE_MACRO(32) \
- CASE_MACRO(64) \
- CASE_MACRO(128)
-
-void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
- constexpr unsigned NumFlags = 2;
- if (N->getNumOperands() < NumFlags + 3)
- report_fatal_error("Malformed spcompress intrinsic");
-
- unsigned FlagOp = N->getNumOperands() - NumFlags;
- unsigned IdxSize =
- cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
- unsigned NumTgt =
- cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
- if (NumTgt != 4)
- report_fatal_error(Twine("Invalid spcompress flags: num_tgt=") +
- Twine(NumTgt));
-
- // The data bundle contains two registers per repetition.
- unsigned DataSize = FlagOp - 2;
- unsigned RepeatFactor = DataSize / 2;
- EVT DataRegVT = N->getOperand(1).getValueType();
- auto ElemSize = getSPDataElemSize(DataRegVT);
- if (!ElemSize || DataSize % 2 != 0 ||
- !nvvm::isValidSPRepeatFactor(RepeatFactor))
- report_fatal_error("spcompress operand/result types do not match "
- "the SP intrinsic flags");
- unsigned ElemSizeValue = *ElemSize;
-
- for (unsigned I = 1; I <= DataSize; ++I)
- if (N->getOperand(I).getValueType() != DataRegVT)
- report_fatal_error("spcompress operand/result types do not match "
- "the SP intrinsic flags");
- if (N->getOperand(FlagOp - 1).getValueType() != MVT::i32)
- report_fatal_error("spcompress operand/result types do not match "
- "the SP intrinsic flags");
-
- auto Layout = nvvm::getSPCompressLayout(ElemSizeValue, IdxSize, RepeatFactor);
- if (!Layout)
- report_fatal_error(Twine("Invalid spcompress flags: elem_size=") +
- Twine(ElemSizeValue) + ", idx_size=" + Twine(IdxSize));
-
- if (N->getNumValues() != Layout->MetadataSize + Layout->CompressedDataSize ||
- N->getNumOperands() != Layout->DataSize + NumFlags + 2)
- report_fatal_error("spcompress operand/result types do not match "
- "the SP intrinsic flags");
- for (unsigned I = 0; I != Layout->MetadataSize; ++I)
- if (N->getValueType(I) != MVT::i32)
- report_fatal_error("spcompress operand/result types do not match "
- "the SP intrinsic flags");
- for (unsigned I = Layout->MetadataSize, E = N->getNumValues(); I != E; ++I)
- if (N->getValueType(I) != DataRegVT)
- report_fatal_error("spcompress operand/result types do not match "
- "the SP intrinsic flags");
-
- SDLoc DL(N);
- SmallVector<SDValue, 16> Operands;
- Operands.reserve(N->getNumOperands() - NumFlags + 2);
- Operands.push_back(getI32Imm(ElemSizeValue, DL));
- Operands.push_back(getI32Imm(IdxSize, DL));
- Operands.push_back(getI32Imm(RepeatFactor, DL));
- for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
- Operands.push_back(N->getOperand(I));
-
-#define REPLACE_SPCOMPRESS_NODE(NumResults) \
+// TableGen's variadic morph encoding supports at most seven fixed operands,
+// while SP register bundles can be much wider, so select their opcodes here.
+static unsigned getSPCompressOpcode(unsigned NumResults) {
+#define SPCOMPRESS_OPCODE_CASE(NumResults) \
case NumResults: \
- ReplaceNode(N, CurDAG->getMachineNode(NVPTX::SPCOMPRESS_OUT_##NumResults, \
- DL, N->getVTList(), Operands)); \
- return;
-
- switch (Layout->MetadataSize + Layout->CompressedDataSize) {
- SPCOMPRESS_OPCODE_CASES(REPLACE_SPCOMPRESS_NODE)
+ return NVPTX::SPCOMPRESS_OUT_##NumResults
+
+ switch (NumResults) {
+ SPCOMPRESS_OPCODE_CASE(2);
+ SPCOMPRESS_OPCODE_CASE(3);
+ SPCOMPRESS_OPCODE_CASE(5);
+ SPCOMPRESS_OPCODE_CASE(6);
+ SPCOMPRESS_OPCODE_CASE(9);
+ SPCOMPRESS_OPCODE_CASE(10);
+ SPCOMPRESS_OPCODE_CASE(12);
+ SPCOMPRESS_OPCODE_CASE(18);
+ SPCOMPRESS_OPCODE_CASE(20);
+ SPCOMPRESS_OPCODE_CASE(24);
+ SPCOMPRESS_OPCODE_CASE(36);
+ SPCOMPRESS_OPCODE_CASE(40);
+ SPCOMPRESS_OPCODE_CASE(48);
+ SPCOMPRESS_OPCODE_CASE(72);
+ SPCOMPRESS_OPCODE_CASE(80);
+ SPCOMPRESS_OPCODE_CASE(96);
default:
llvm_unreachable("invalid spcompress result count");
}
-#undef REPLACE_SPCOMPRESS_NODE
-}
-
-void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N) {
- // NumSrc is inferred from the logical vector lengths before register
- // padding and appended as an internal operand by lowerSPDecompress.
- constexpr unsigned NumFlags = 3;
- if (N->getNumOperands() < NumFlags + 3)
- report_fatal_error("Malformed spdecompress intrinsic");
-
- unsigned FlagOp = N->getNumOperands() - NumFlags;
- unsigned IdxSize =
- cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
- unsigned NumTgt =
- cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
- unsigned NumSrc =
- cast<ConstantSDNode>(N->getOperand(FlagOp + 2))->getZExtValue();
- EVT DataRegVT = N->getValueType(0);
- auto ElemSize = getSPDataElemSize(DataRegVT);
- if (!ElemSize)
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
- unsigned ElemSizeValue = *ElemSize;
-
- // data_size = num_tgt * elem_size * repeat_factor / 32.
- unsigned RepeatFactorDenominator = NumTgt * ElemSizeValue;
- unsigned RepeatFactorNumerator = N->getNumValues() * 32;
- if (RepeatFactorDenominator == 0 ||
- RepeatFactorNumerator % RepeatFactorDenominator != 0)
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
- unsigned RepeatFactor = RepeatFactorNumerator / RepeatFactorDenominator;
- if (!nvvm::isValidSPRepeatFactor(RepeatFactor))
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
-
- auto Layout = nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSizeValue,
- IdxSize, RepeatFactor);
- if (!Layout)
- report_fatal_error(Twine("Invalid spdecompress flags: num_src=") +
- Twine(NumSrc) + ", num_tgt=" + Twine(NumTgt) +
- ", elem_size=" + Twine(ElemSizeValue) +
- ", idx_size=" + Twine(IdxSize));
-
- if (N->getNumValues() != Layout->DataSize ||
- N->getNumOperands() !=
- Layout->MetadataSize + Layout->CompressedDataSize + NumFlags + 1)
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
- for (unsigned I = 1; I <= Layout->MetadataSize; ++I)
- if (N->getOperand(I).getValueType() != MVT::i32)
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
- for (unsigned I = Layout->MetadataSize + 1; I != FlagOp; ++I)
- if (N->getOperand(I).getValueType() != DataRegVT)
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
-
- SDLoc DL(N);
- SmallVector<SDValue, 16> Operands;
- Operands.reserve(N->getNumOperands() - NumFlags + 4);
- Operands.push_back(getI32Imm(ElemSizeValue, DL));
- Operands.push_back(getI32Imm(IdxSize, DL));
- Operands.push_back(getI32Imm(RepeatFactor, DL));
- Operands.push_back(getI32Imm(NumSrc, DL));
- Operands.push_back(getI32Imm(NumTgt, DL));
- for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
- Operands.push_back(N->getOperand(I));
+#undef SPCOMPRESS_OPCODE_CASE
+}
-#define REPLACE_SPDECOMPRESS_NODE(NumResults) \
+static unsigned getSPDecompressOpcode(unsigned NumResults) {
+#define SPDECOMPRESS_OPCODE_CASE(NumResults) \
case NumResults: \
- ReplaceNode(N, \
- CurDAG->getMachineNode(NVPTX::SPDECOMPRESS_OUT_##NumResults, \
- DL, N->getVTList(), Operands)); \
- return;
-
- switch (Layout->DataSize) {
- SPDECOMPRESS_OPCODE_CASES(REPLACE_SPDECOMPRESS_NODE)
+ return NVPTX::SPDECOMPRESS_OUT_##NumResults
+
+ switch (NumResults) {
+ SPDECOMPRESS_OPCODE_CASE(1);
+ SPDECOMPRESS_OPCODE_CASE(2);
+ SPDECOMPRESS_OPCODE_CASE(4);
+ SPDECOMPRESS_OPCODE_CASE(8);
+ SPDECOMPRESS_OPCODE_CASE(16);
+ SPDECOMPRESS_OPCODE_CASE(32);
+ SPDECOMPRESS_OPCODE_CASE(64);
+ SPDECOMPRESS_OPCODE_CASE(128);
default:
llvm_unreachable("invalid spdecompress result count");
}
-#undef REPLACE_SPDECOMPRESS_NODE
+#undef SPDECOMPRESS_OPCODE_CASE
}
-#undef SPDECOMPRESS_OPCODE_CASES
-#undef SPCOMPRESS_OPCODE_CASES
-
-bool NVPTXDAGToDAGISel::tryIntrinsicNoChain(SDNode *N) {
- unsigned IID = N->getConstantOperandVal(0);
- switch (IID) {
- default:
- return false;
- case Intrinsic::nvvm_spcompress:
- selectSPCompress(N);
- return true;
- case Intrinsic::nvvm_spdecompress:
- selectSPDecompress(N);
- return true;
- }
+void NVPTXDAGToDAGISel::selectSPNode(SDNode *N) {
+ unsigned Opcode = N->getOpcode() == NVPTXISD::SPCOMPRESS
+ ? getSPCompressOpcode(N->getNumValues())
+ : getSPDecompressOpcode(N->getNumValues());
+ SmallVector<SDValue, 16> Operands;
+ llvm::append_range(Operands, N->ops());
+ ReplaceNode(
+ N, CurDAG->getMachineNode(Opcode, SDLoc(N), N->getVTList(), Operands));
}
bool NVPTXDAGToDAGISel::tryIntrinsicChain(SDNode *N) {
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
index 0700eb841ffe4b..0a9ea79315d882 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
@@ -3219,17 +3219,6 @@ static unsigned getSPVectorNumParts(EVT VT) {
return divideCeil(VT.getVectorNumElements(), PartElts);
}
-static bool needsSPVectorLowering(SDNode *N) {
- for (EVT VT : N->values())
- if (VT.isVector() && VT != getSPVectorPartType(VT))
- return true;
- for (const SDValue &Operand : N->ops())
- if (EVT VT = Operand.getValueType();
- VT.isVector() && VT != getSPVectorPartType(VT))
- return true;
- return false;
-}
-
static void splitSPVector(SDValue Vector, SmallVectorImpl<SDValue> &Parts,
SelectionDAG &DAG) {
EVT VT = Vector.getValueType();
@@ -3249,7 +3238,7 @@ static void splitSPVector(SDValue Vector, SmallVectorImpl<SDValue> &Parts,
DAG.getUNDEF(PaddedVT), Vector,
DAG.getVectorIdxConstant(0, SDLoc(Vector)));
}
- for (unsigned I = 0; I != NumParts; ++I)
+ for (unsigned I : llvm::seq(NumParts))
Parts.push_back(
DAG.getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(Vector), PartVT, Vector,
DAG.getVectorIdxConstant(I * PartElts, SDLoc(Vector))));
@@ -3259,10 +3248,16 @@ static SDValue joinSPVector(EVT VT, ArrayRef<SDValue> Parts, const SDLoc &DL,
SelectionDAG &DAG) {
assert(Parts.size() == getSPVectorNumParts(VT) &&
"incorrect number of SP vector parts");
+ EVT PartVT = getSPVectorPartType(VT);
+ SmallVector<SDValue> TypedParts;
+ for (SDValue Part : Parts)
+ TypedParts.push_back(
+ Part.getValueType() == PartVT ? Part : DAG.getBitcast(PartVT, Part));
+ Parts = TypedParts;
+
if (Parts.size() == 1 && Parts.front().getValueType() == VT)
return Parts.front();
if (Parts.front().getValueType().isVector()) {
- EVT PartVT = Parts.front().getValueType();
EVT JoinedVT =
EVT::getVectorVT(*DAG.getContext(), VT.getVectorElementType(),
Parts.size() * PartVT.getVectorNumElements());
@@ -3277,37 +3272,57 @@ static SDValue joinSPVector(EVT VT, ArrayRef<SDValue> Parts, const SDLoc &DL,
return DAG.getBuildVector(VT, DL, Parts);
}
-static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
+static void appendSplitSPOperands(ArrayRef<SDUse> Input,
+ SmallVectorImpl<SDValue> &Output,
+ SelectionDAG &DAG) {
+ for (const SDValue &Operand : Input) {
+ if (!Operand.getValueType().isVector()) {
+ Output.push_back(Operand);
+ continue;
+ }
+
+ SmallVector<SDValue> Parts;
+ splitSPVector(Operand, Parts, DAG);
+ for (SDValue Part : Parts)
+ Output.push_back(Part.getValueType() == MVT::i32
+ ? Part
+ : DAG.getBitcast(MVT::i32, Part));
+ }
+}
+
+static SDValue lowerSPIntrinsic(SDValue Op, ArrayRef<SDValue> Qualifiers,
+ SelectionDAG &DAG) {
SDNode *N = Op.getNode();
SDLoc DL(N);
- if (!needsSPVectorLowering(N))
- return Op;
-
SmallVector<SDValue, 16> Ops;
- for (const SDValue &Operand : N->ops()) {
- if (Operand.getValueType().isVector())
- splitSPVector(Operand, Ops, DAG);
- else
- Ops.push_back(Operand);
- }
+ llvm::append_range(Ops, Qualifiers);
+ appendSplitSPOperands(N->ops().slice(1, 2), Ops, DAG);
SmallVector<EVT, 16> ResultTys;
for (const EVT VT : N->values()) {
if (VT.isVector())
- ResultTys.append(getSPVectorNumParts(VT), getSPVectorPartType(VT));
+ ResultTys.append(getSPVectorNumParts(VT), MVT::i32);
else
ResultTys.push_back(VT);
}
- SDValue Lowered = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResultTys, Ops);
+ auto IID = static_cast<Intrinsic::ID>(N->getConstantOperandVal(0));
+ assert((IID == Intrinsic::nvvm_spcompress ||
+ IID == Intrinsic::nvvm_spdecompress) &&
+ "unexpected SP intrinsic");
+ unsigned Opcode = IID == Intrinsic::nvvm_spcompress ? NVPTXISD::SPCOMPRESS
+ : NVPTXISD::SPDECOMPRESS;
+ SDValue Lowered = DAG.getNode(Opcode, DL, ResultTys, Ops);
SmallVector<SDValue, 2> Retvals;
- for (unsigned NewI = 0, I = 0, E = N->getNumValues(); I != E; ++I) {
+ unsigned NewI = 0;
+ for (unsigned I : llvm::seq(N->getNumValues())) {
if (EVT VT = N->getValueType(I); VT.isVector()) {
SmallVector<SDValue> Parts;
- for (auto NewE = NewI + getSPVectorNumParts(VT); NewI != NewE; ++NewI)
- Parts.push_back(Lowered.getValue(NewI));
+ for (unsigned Part : llvm::seq(getSPVectorNumParts(VT)))
+ Parts.push_back(Lowered.getValue(NewI + Part));
+ NewI += Parts.size();
Retvals.push_back(joinSPVector(VT, Parts, DL, DAG));
} else {
Retvals.push_back(Lowered.getValue(NewI));
@@ -3318,55 +3333,54 @@ static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
return DAG.getMergeValues(Retvals, DL);
}
+static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
+ SDNode *N = Op.getNode();
+ SDLoc DL(N);
+ EVT DataVT = N->getOperand(1).getValueType();
+ unsigned ElemSize = DataVT.getScalarSizeInBits();
+ unsigned IdxSize = cast<ConstantSDNode>(N->getOperand(3))->getZExtValue();
+ unsigned DataSize = getSPVectorNumParts(DataVT);
+ assert(DataSize % 2 == 0 && "invalid spcompress data size");
+ unsigned RepeatFactor = DataSize / 2;
+ assert(nvvm::isValidSPRepeatFactor(RepeatFactor) &&
+ "invalid spcompress repeat factor");
+
+ SmallVector<SDValue, 3> Qualifiers = {
+ DAG.getTargetConstant(ElemSize, DL, MVT::i32),
+ DAG.getTargetConstant(IdxSize, DL, MVT::i32),
+ DAG.getTargetConstant(RepeatFactor, DL, MVT::i32)};
+ return lowerSPIntrinsic(Op, Qualifiers, DAG);
+}
+
static SDValue lowerSPDecompress(SDValue Op, SelectionDAG &DAG) {
SDNode *N = Op.getNode();
SDLoc DL(N);
- // A lowered node has the inferred num_src operand appended and may have more
- // than one operand for each register bundle.
- if (N->getNumOperands() != 5)
- return Op;
EVT ResVT0 = N->getValueType(0);
EVT CDataVT = N->getOperand(2).getValueType();
- if (!ResVT0.isVector() || !CDataVT.isVector())
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
+ assert(ResVT0.isVector() && CDataVT.isVector() &&
+ "invalid spdecompress types");
unsigned NumTgt = cast<ConstantSDNode>(N->getOperand(4))->getZExtValue();
unsigned DataElts = ResVT0.getVectorNumElements();
- if (NumTgt == 0 || DataElts % NumTgt != 0)
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
+ assert(NumTgt != 0 && DataElts % NumTgt == 0 &&
+ "invalid spdecompress target group size");
unsigned RepeatFactor = DataElts / NumTgt;
unsigned CDataElts = CDataVT.getVectorNumElements();
- if (RepeatFactor == 0 || CDataElts % RepeatFactor != 0)
- report_fatal_error("spdecompress operand/result types do not match "
- "the SP intrinsic flags");
+ assert(nvvm::isValidSPRepeatFactor(RepeatFactor) &&
+ CDataElts % RepeatFactor == 0 &&
+ "invalid spdecompress source group size");
unsigned NumSrc = CDataElts / RepeatFactor;
- unsigned NumParts = getSPVectorNumParts(ResVT0);
- EVT PartVT = getSPVectorPartType(ResVT0);
- SmallVector<EVT, 16> ListVTs(NumParts, PartVT);
- SDVTList ResVTs = DAG.getVTList(ListVTs);
-
- SmallVector<SDValue, 16> Ops;
- Ops.push_back(N->getOperand(0));
-
- for (unsigned I = 1; I < N->getNumOperands(); ++I) {
- SDValue Operand = N->getOperand(I);
- if (Operand.getValueType().isVector())
- splitSPVector(Operand, Ops, DAG);
- else
- Ops.push_back(Operand);
- }
- Ops.push_back(DAG.getConstant(NumSrc, DL, MVT::i32));
-
- SDValue NewNode = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResVTs, Ops);
-
- SmallVector<SDValue, 16> Parts;
- for (unsigned I = 0; I < NumParts; ++I)
- Parts.push_back(NewNode.getValue(I));
- return joinSPVector(ResVT0, Parts, DL, DAG);
+ unsigned ElemSize = ResVT0.getScalarSizeInBits();
+ unsigned IdxSize = cast<ConstantSDNode>(N->getOperand(3))->getZExtValue();
+ SmallVector<SDValue, 5> Qualifiers = {
+ DAG.getTargetConstant(ElemSize, DL, MVT::i32),
+ DAG.getTargetConstant(IdxSize, DL, MVT::i32),
+ DAG.getTargetConstant(RepeatFactor, DL, MVT::i32),
+ DAG.getTargetConstant(NumSrc, DL, MVT::i32),
+ DAG.getTargetConstant(NumTgt, DL, MVT::i32)};
+ return lowerSPIntrinsic(Op, Qualifiers, DAG);
}
static SDValue lowerIntrinsicWOChain(SDValue Op, SelectionDAG &DAG) {
@@ -7467,16 +7481,9 @@ static SDValue combineIntrinsicWOChain(SDNode *N,
return combineFAddWithNeg(N, DCI.DAG, IID, RoundingMode);
}
case Intrinsic::nvvm_spcompress:
- if (needsSPVectorLowering(N))
- return lowerSPCompress(SDValue(N, 0), DCI.DAG);
- break;
- case Intrinsic::nvvm_spdecompress: {
- // The IR intrinsic has the ID plus four arguments; lowered nodes also have
- // an inferred num_src operand, so do not lower them again.
- if (N->getNumOperands() == 5)
- return lowerSPDecompress(SDValue(N, 0), DCI.DAG);
- break;
- }
+ return lowerSPCompress(SDValue(N, 0), DCI.DAG);
+ case Intrinsic::nvvm_spdecompress:
+ return lowerSPDecompress(SDValue(N, 0), DCI.DAG);
}
return SDValue();
}
diff --git a/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.cpp b/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.cpp
index 710d063e75725c..c828747ce95451 100644
--- a/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.cpp
@@ -39,6 +39,8 @@ const char *NVPTXSelectionDAGInfo::getTargetNodeName(unsigned Opcode) const {
MAKE_CASE(NVPTXISD::SETP_F16X2)
MAKE_CASE(NVPTXISD::SETP_BF16X2)
MAKE_CASE(NVPTXISD::UNPACK_VECTOR)
+ MAKE_CASE(NVPTXISD::SPCOMPRESS)
+ MAKE_CASE(NVPTXISD::SPDECOMPRESS)
}
#undef MAKE_CASE
@@ -62,6 +64,10 @@ void NVPTXSelectionDAGInfo::verifyTargetNode(const SelectionDAG &DAG,
case NVPTXISD::ProxyReg:
// invalid number of results; expected 2, got 1
return;
+ case NVPTXISD::SPCOMPRESS:
+ case NVPTXISD::SPDECOMPRESS:
+ // The packed register lists have variable numbers of operands and results.
+ return;
}
return SelectionDAGGenTargetInfo::verifyTargetNode(DAG, N);
diff --git a/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h b/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h
index 9dd0a1eaa5856b..8e29d08817e835 100644
--- a/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h
+++ b/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h
@@ -21,6 +21,8 @@ enum NodeType : unsigned {
SETP_F16X2 = GENERATED_OPCODE_END,
SETP_BF16X2,
UNPACK_VECTOR,
+ SPCOMPRESS,
+ SPDECOMPRESS,
FIRST_MEMORY_OPCODE,
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
index fd4340338f9913..8120fd97b24ae0 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
@@ -1,18 +1,19 @@
; RUN: split-file %s %t
-; RUN: not --crash llc < %t/bad-elem-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
+; RUN: not llvm-as -disable-output < %t/bad-elem-type.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
; RUN: not llvm-as -disable-output < %t/bad-num-tgt.ll 2>&1 | FileCheck %s --check-prefix=BAD-NUM-TGT
-; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
-; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
-; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
-; RUN: not --crash llc < %t/bad-result-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
-; RUN: not --crash llc < %t/bad-data-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA-TY
+; RUN: not llvm-as -disable-output < %t/bad-mdata-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
+; RUN: not llvm-as -disable-output < %t/bad-cdata-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
+; RUN: not llvm-as -disable-output < %t/bad-data-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-DATA
+; RUN: not llvm-as -disable-output < %t/bad-ratio.ll 2>&1 | FileCheck %s --check-prefix=BAD-RATIO
+; RUN: not llvm-as -disable-output < %t/bad-result-type.ll 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
+; RUN: not llvm-as -disable-output < %t/bad-data-type.ll 2>&1 | FileCheck %s --check-prefix=BAD-DATA-TY
;--- bad-elem-type.ll
define { i32, <2 x i32> } @bad_spcompress_elem_type(i32 %spdesc,
<2 x i32> %data) {
-; BAD-ELEM: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+; BAD-ELEM: invalid llvm.nvvm.spcompress operand or result type
%res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.i32.v2i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 2, i32 4)
ret { i32, <2 x i32> } %res
}
@@ -39,7 +40,7 @@ define { i32, <4 x i8> } @bad_spcompress_num_tgt(i32 %spdesc,
define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
<32 x i8> %data) {
-; BAD-MDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+; BAD-MDATA: invalid llvm.nvvm.spcompress layout
%res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4, i32 4)
%mdata = extractvalue { i32, <16 x i8> } %res, 0
ret i32 %mdata
@@ -49,7 +50,7 @@ define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
define <8 x i8> @bad_spcompress_compressed_data_register_count(
i32 %spdesc, <32 x i8> %data) {
-; BAD-CDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+; BAD-CDATA: invalid llvm.nvvm.spcompress layout
%res = call { <2 x i32>, <8 x i8> } @llvm.nvvm.spcompress.v2i32.v8i8.v32i8(<32 x i8> %data, i32 %spdesc, i32 4, i32 4)
%cdata = extractvalue { <2 x i32>, <8 x i8> } %res, 1
ret <8 x i8> %cdata
@@ -59,16 +60,25 @@ define <8 x i8> @bad_spcompress_compressed_data_register_count(
define <16 x i8> @bad_spcompress_data_register_count(i32 %spdesc,
<24 x i8> %data) {
-; BAD-DATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+; BAD-DATA: invalid llvm.nvvm.spcompress layout
%res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.v2i32.v16i8.v24i8(<24 x i8> %data, i32 %spdesc, i32 4, i32 4)
%cdata = extractvalue { <2 x i32>, <16 x i8> } %res, 1
ret <16 x i8> %cdata
}
+;--- bad-ratio.ll
+
+define { i32, <3 x i8> } @bad_spcompress_ratio(i32 %spdesc,
+ <8 x i8> %data) {
+; BAD-RATIO: invalid llvm.nvvm.spcompress layout
+ %res = call { i32, <3 x i8> } @llvm.nvvm.spcompress.i32.v3i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2, i32 4)
+ ret { i32, <3 x i8> } %res
+}
+
;--- bad-result-type.ll
define i64 @bad_spcompress_result_type(i32 %spdesc, <8 x i8> %data) {
-; BAD-RESULT-TY: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+; BAD-RESULT-TY: invalid llvm.nvvm.spcompress operand or result type
%res = call { i64, <4 x i8> } @llvm.nvvm.spcompress.i64.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, i32 2, i32 4)
%mdata = extractvalue { i64, <4 x i8> } %res, 0
ret i64 %mdata
@@ -78,7 +88,7 @@ define i64 @bad_spcompress_result_type(i32 %spdesc, <8 x i8> %data) {
define { i32, <2 x i16> } @bad_spcompress_data_type(i32 %spdesc,
<8 x i8> %data) {
-; BAD-DATA-TY: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+; BAD-DATA-TY: invalid llvm.nvvm.spcompress operand or result type
%res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.i32.v2i16.v8i8(<8 x i8> %data, i32 %spdesc, i32 2, i32 4)
ret { i32, <2 x i16> } %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
index 23d9d7a8d2b9bc..6dd381d767b2b0 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
@@ -5,7 +5,7 @@
define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <8 x i8> %data) {
; FORMAT-LABEL: define { i32, <4 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -23,7 +23,7 @@ define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <8
define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <4 x i16> %data) {
; FORMAT-LABEL: define { i32, <2 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -41,7 +41,7 @@ define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <
define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <8 x i8> %data) {
; FORMAT-LABEL: define { i32, <4 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <4 x i8> } @llvm.nvvm.spcompress.i32.v4i8.v8i8(<8 x i8> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -59,7 +59,7 @@ define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <8
define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <4 x i16> %data) {
; FORMAT-LABEL: define { i32, <2 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <2 x i16> } @llvm.nvvm.spcompress.i32.v2i16.v4i16(<4 x i16> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<6>;
@@ -77,7 +77,7 @@ define {i32, <2 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <
define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <16 x i8> %data) {
; FORMAT-LABEL: define { i32, <8 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -95,7 +95,7 @@ define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <16
define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <8 x i16> %data) {
; FORMAT-LABEL: define { i32, <4 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -113,7 +113,7 @@ define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <
define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <16 x i8> %data) {
; FORMAT-LABEL: define { i32, <8 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <8 x i8> } @llvm.nvvm.spcompress.i32.v8i8.v16i8(<16 x i8> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -131,7 +131,7 @@ define {i32, <8 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <16
define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <8 x i16> %data) {
; FORMAT-LABEL: define { i32, <4 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <4 x i16> } @llvm.nvvm.spcompress.i32.v4i16.v8i16(<8 x i16> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<9>;
@@ -149,7 +149,7 @@ define {i32, <4 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <
define {i32, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <32 x i8> %data) {
; FORMAT-LABEL: define { i32, <16 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <16 x i8> } @llvm.nvvm.spcompress.i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -168,7 +168,7 @@ define {i32, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <3
define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <16 x i16> %data) {
; FORMAT-LABEL: define { i32, <8 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -187,7 +187,7 @@ define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <
define {<2 x i32>, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spdesc, <32 x i8> %data) {
; FORMAT-LABEL: define { <2 x i32>, <16 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.v2i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i8> } @llvm.nvvm.spcompress.v2i32.v16i8.v32i8(<32 x i8> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<16>;
@@ -206,7 +206,7 @@ define {<2 x i32>, <16 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spde
define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <16 x i16> %data) {
; FORMAT-LABEL: define { i32, <8 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <8 x i16> } @llvm.nvvm.spcompress.i32.v8i16.v16i16(<16 x i16> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -225,7 +225,7 @@ define {i32, <8 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <
define {<2 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spdesc, <64 x i8> %data) {
; FORMAT-LABEL: define { <2 x i32>, <32 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i8> } @llvm.nvvm.spcompress.v2i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i8> } @llvm.nvvm.spcompress.v2i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<28>;
@@ -247,7 +247,7 @@ define {<2 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spde
define {i32, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <32 x i16> %data) {
; FORMAT-LABEL: define { i32, <16 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call { i32, <16 x i16> } @llvm.nvvm.spcompress.i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { i32, <16 x i16> } @llvm.nvvm.spcompress.i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -269,7 +269,7 @@ define {i32, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc,
define {<4 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spdesc, <64 x i8> %data) {
; FORMAT-LABEL: define { <4 x i32>, <32 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i8> } @llvm.nvvm.spcompress.v4i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i8> } @llvm.nvvm.spcompress.v4i32.v32i8.v64i8(<64 x i8> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<30>;
@@ -291,7 +291,7 @@ define {<4 x i32>, <32 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spde
define {<2 x i32>, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spdesc, <32 x i16> %data) {
; FORMAT-LABEL: define { <2 x i32>, <16 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i16> } @llvm.nvvm.spcompress.v2i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i16> } @llvm.nvvm.spcompress.v2i32.v16i16.v32i16(<32 x i16> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<28>;
@@ -313,7 +313,7 @@ define {<2 x i32>, <16 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %sp
define {<4 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spdesc, <128 x i8> %data) {
; FORMAT-LABEL: define { <4 x i32>, <64 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i8> } @llvm.nvvm.spcompress.v4i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i8> } @llvm.nvvm.spcompress.v4i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<54>;
@@ -341,7 +341,7 @@ define {<4 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spd
define {<2 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %spdesc, <64 x i16> %data) {
; FORMAT-LABEL: define { <2 x i32>, <32 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i16> } @llvm.nvvm.spcompress.v2i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <2 x i32>, <32 x i16> } @llvm.nvvm.spcompress.v2i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<52>;
@@ -369,7 +369,7 @@ define {<2 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %s
define {<8 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spdesc, <128 x i8> %data) {
; FORMAT-LABEL: define { <8 x i32>, <64 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i8> } @llvm.nvvm.spcompress.v8i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i8> } @llvm.nvvm.spcompress.v8i32.v64i8.v128i8(<128 x i8> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<58>;
@@ -398,7 +398,7 @@ define {<8 x i32>, <64 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spd
define {<4 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %spdesc, <64 x i16> %data) {
; FORMAT-LABEL: define { <4 x i32>, <32 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i16> } @llvm.nvvm.spcompress.v4i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i16> } @llvm.nvvm.spcompress.v4i32.v32i16.v64i16(<64 x i16> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<54>;
@@ -426,7 +426,7 @@ define {<4 x i32>, <32 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %s
define {<8 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %spdesc, <256 x i8> %data) {
; FORMAT-LABEL: define { <8 x i32>, <128 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i8> } @llvm.nvvm.spcompress.v8i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i8> } @llvm.nvvm.spcompress.v8i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<106>;
@@ -467,7 +467,7 @@ define {<8 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %sp
define {<4 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %spdesc, <128 x i16> %data) {
; FORMAT-LABEL: define { <4 x i32>, <64 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i16> } @llvm.nvvm.spcompress.v4i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <4 x i32>, <64 x i16> } @llvm.nvvm.spcompress.v4i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<102>;
@@ -507,7 +507,7 @@ define {<4 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %s
define {<16 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %spdesc, <256 x i8> %data) {
; FORMAT-LABEL: define { <16 x i32>, <128 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i8> } @llvm.nvvm.spcompress.v16i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i8> } @llvm.nvvm.spcompress.v16i32.v128i8.v256i8(<256 x i8> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<114>;
@@ -550,7 +550,7 @@ define {<16 x i32>, <128 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %s
define {<8 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %spdesc, <128 x i16> %data) {
; FORMAT-LABEL: define { <8 x i32>, <64 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i16> } @llvm.nvvm.spcompress.v8i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i16> } @llvm.nvvm.spcompress.v8i32.v64i16.v128i16(<128 x i16> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<106>;
@@ -591,7 +591,7 @@ define {<8 x i32>, <64 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %s
define {<16 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %spdesc, <512 x i8> %data) {
; FORMAT-LABEL: define { <16 x i32>, <256 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call { <16 x i32>, <256 x i8> } @llvm.nvvm.spcompress.v16i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <16 x i32>, <256 x i8> } @llvm.nvvm.spcompress.v16i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<210>;
@@ -658,7 +658,7 @@ define {<16 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %s
define {<8 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %spdesc, <256 x i16> %data) {
; FORMAT-LABEL: define { <8 x i32>, <128 x i16> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i16> } @llvm.nvvm.spcompress.v8i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <8 x i32>, <128 x i16> } @llvm.nvvm.spcompress.v8i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<202>;
@@ -723,7 +723,7 @@ define {<8 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %
define {<32 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %spdesc, <512 x i8> %data) {
; FORMAT-LABEL: define { <32 x i32>, <256 x i8> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call { <32 x i32>, <256 x i8> } @llvm.nvvm.spcompress.v32i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <32 x i32>, <256 x i8> } @llvm.nvvm.spcompress.v32i32.v256i8.v512i8(<512 x i8> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<226>;
@@ -794,7 +794,7 @@ define {<32 x i32>, <256 x i8>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %s
define {<16 x i32>, <128 x i16>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %spdesc, <256 x i16> %data) {
; FORMAT-LABEL: define { <16 x i32>, <128 x i16> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i16> } @llvm.nvvm.spcompress.v16i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call { <16 x i32>, <128 x i16> } @llvm.nvvm.spcompress.v16i32.v128i16.v256i16(<256 x i16> %data, i32 %spdesc, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<210>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
index d4e1b743363875..75956faa20d179 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
@@ -1,20 +1,20 @@
; RUN: split-file %s %t
-; RUN: not --crash llc < %t/bad-elem-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
+; RUN: not llvm-as -disable-output < %t/bad-elem-type.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
; RUN: not llvm-as -disable-output < %t/bad-num-tgt.ll 2>&1 | FileCheck %s --check-prefix=BAD-NUM-TGT
-; RUN: not --crash llc < %t/bad-idx-size-for-target.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-IDX-TARGET
-; RUN: not --crash llc < %t/bad-input-span.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-INPUT-SPAN
-; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
-; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
-; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
-; RUN: not --crash llc < %t/bad-result-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
-; RUN: not --crash llc < %t/bad-mdata-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA-TY
+; RUN: not llvm-as -disable-output < %t/bad-idx-size-for-target.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX-TARGET
+; RUN: not llvm-as -disable-output < %t/bad-input-span.ll 2>&1 | FileCheck %s --check-prefix=BAD-INPUT-SPAN
+; RUN: not llvm-as -disable-output < %t/bad-mdata-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
+; RUN: not llvm-as -disable-output < %t/bad-cdata-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
+; RUN: not llvm-as -disable-output < %t/bad-data-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-DATA
+; RUN: not llvm-as -disable-output < %t/bad-result-type.ll 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
+; RUN: not llvm-as -disable-output < %t/bad-mdata-type.ll 2>&1 | FileCheck %s --check-prefix=BAD-MDATA-TY
;--- bad-elem-type.ll
define <2 x i32> @bad_spdecompress_elem_type(i32 %metadata,
<2 x i32> %compdata) {
-; BAD-ELEM: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+; BAD-ELEM: invalid llvm.nvvm.spdecompress operand or result type
%res = call <2 x i32> @llvm.nvvm.spdecompress.v2i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 2, i32 4)
ret <2 x i32> %res
}
@@ -41,7 +41,7 @@ define <4 x i8> @bad_spdecompress_num_tgt(i32 %metadata,
define <32 x i8> @bad_spdecompress_idx_size_for_target(
i32 %metadata, <4 x i8> %compdata) {
-; BAD-IDX-TARGET: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=8, elem_size=8, idx_size=2
+; BAD-IDX-TARGET: invalid llvm.nvvm.spdecompress layout
%res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, i32 2, i32 8)
ret <32 x i8> %res
}
@@ -50,7 +50,7 @@ define <32 x i8> @bad_spdecompress_idx_size_for_target(
define <8 x i16> @bad_spdecompress_input_span(<2 x i32> %metadata,
<4 x i16> %compdata) {
-; BAD-INPUT-SPAN: LLVM ERROR: Invalid spdecompress flags: num_src=4, num_tgt=8, elem_size=16, idx_size=4
+; BAD-INPUT-SPAN: invalid llvm.nvvm.spdecompress layout
%res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.v2i32.v4i16(<2 x i32> %metadata, <4 x i16> %compdata, i32 4, i32 8)
ret <8 x i16> %res
}
@@ -59,7 +59,7 @@ define <8 x i16> @bad_spdecompress_input_span(<2 x i32> %metadata,
define <16 x i8> @bad_spdecompress_metadata_register_count(
<2 x i32> %metadata, <8 x i8> %compdata) {
-; BAD-MDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+; BAD-MDATA: invalid llvm.nvvm.spdecompress layout
%res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.v2i32.v8i8(<2 x i32> %metadata, <8 x i8> %compdata, i32 4, i32 4)
ret <16 x i8> %res
}
@@ -68,7 +68,7 @@ define <16 x i8> @bad_spdecompress_metadata_register_count(
define <8 x i16> @bad_spdecompress_compressed_data_register_count(
i32 %metadata, <6 x i16> %compdata) {
-; BAD-CDATA: LLVM ERROR: Invalid spdecompress flags: num_src=3, num_tgt=4, elem_size=16, idx_size=4
+; BAD-CDATA: invalid llvm.nvvm.spdecompress layout
%res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v6i16(i32 %metadata, <6 x i16> %compdata, i32 4, i32 4)
ret <8 x i16> %res
}
@@ -77,7 +77,7 @@ define <8 x i16> @bad_spdecompress_compressed_data_register_count(
define <12 x i8> @bad_spdecompress_data_register_count(
i32 %metadata, <8 x i8> %compdata) {
-; BAD-DATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+; BAD-DATA: invalid llvm.nvvm.spdecompress layout
%res = call <12 x i8> @llvm.nvvm.spdecompress.v12i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, i32 4, i32 4)
ret <12 x i8> %res
}
@@ -86,7 +86,7 @@ define <12 x i8> @bad_spdecompress_data_register_count(
define <4 x i16> @bad_spdecompress_result_type(i32 %metadata,
<2 x i8> %compdata) {
-; BAD-RESULT-TY: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+; BAD-RESULT-TY: invalid llvm.nvvm.spdecompress operand or result type
%res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 4)
ret <4 x i16> %res
}
@@ -95,7 +95,7 @@ define <4 x i16> @bad_spdecompress_result_type(i32 %metadata,
define <4 x i8> @bad_spdecompress_metadata_type(i64 %metadata,
<2 x i8> %compdata) {
-; BAD-MDATA-TY: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+; BAD-MDATA-TY: invalid llvm.nvvm.spdecompress operand or result type
%res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i64.v2i8(i64 %metadata, <2 x i8> %compdata, i32 2, i32 4)
ret <4 x i8> %res
}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
index 15aec00fce07a5..e4a1302cf2c749 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4
define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <1 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<13>;
@@ -26,7 +26,7 @@ define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -50,7 +50,7 @@ define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
define <64 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<19>;
@@ -70,7 +70,7 @@ define <64 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadat
define <128 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<36>;
@@ -94,7 +94,7 @@ define <128 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metada
define <256 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<71>;
@@ -126,7 +126,7 @@ define <256 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32>
define <512 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<141>;
@@ -176,7 +176,7 @@ define <512 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32>
; elem_size=16, idx_size=4
define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -196,7 +196,7 @@ define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metad
define <32 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<19>;
@@ -216,7 +216,7 @@ define <32 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metad
define <64 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<36>;
@@ -240,7 +240,7 @@ define <64 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metad
define <128 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<70>;
@@ -272,7 +272,7 @@ define <128 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %meta
define <256 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<139>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
index 594c8a25bdaf34..65f5a6e3074b02 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
@@ -5,7 +5,7 @@
define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -28,7 +28,7 @@ define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata,
define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -45,7 +45,7 @@ define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata,
define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -62,7 +62,7 @@ define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -80,7 +80,7 @@ define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadat
define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -101,7 +101,7 @@ define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %m
define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -128,7 +128,7 @@ define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %
define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -151,7 +151,7 @@ define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata,
define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -168,7 +168,7 @@ define <8 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata,
define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -185,7 +185,7 @@ define <16 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -203,7 +203,7 @@ define <32 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %m
define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -224,7 +224,7 @@ define <64 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %m
define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -252,7 +252,7 @@ define <128 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -271,7 +271,7 @@ define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -288,7 +288,7 @@ define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -305,7 +305,7 @@ define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadat
define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -323,7 +323,7 @@ define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<26>;
@@ -344,7 +344,7 @@ define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metad
define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<51>;
@@ -371,7 +371,7 @@ define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32>
define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<101>;
@@ -410,7 +410,7 @@ define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32>
define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -429,7 +429,7 @@ define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -446,7 +446,7 @@ define <4 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -463,7 +463,7 @@ define <8 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadat
define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -481,7 +481,7 @@ define <16 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -502,7 +502,7 @@ define <32 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -529,7 +529,7 @@ define <64 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 2)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
index 9ff415c9988149..926f2186d4649d 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=2
define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <1 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<10>;
@@ -26,7 +26,7 @@ define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata,
define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -49,7 +49,7 @@ define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata,
define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -66,7 +66,7 @@ define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -84,7 +84,7 @@ define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -104,7 +104,7 @@ define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadat
define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -129,7 +129,7 @@ define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %
define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -165,7 +165,7 @@ define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %
; elem_size=8, idx_size=4
define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <1 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<10>;
@@ -185,7 +185,7 @@ define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -208,7 +208,7 @@ define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata,
define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -225,7 +225,7 @@ define <16 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -243,7 +243,7 @@ define <32 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -263,7 +263,7 @@ define <64 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %m
define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -288,7 +288,7 @@ define <128 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -325,7 +325,7 @@ define <256 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; elem_size=16, idx_size=2
define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -344,7 +344,7 @@ define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -361,7 +361,7 @@ define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -379,7 +379,7 @@ define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metada
define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -399,7 +399,7 @@ define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<42>;
@@ -424,7 +424,7 @@ define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metad
define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<83>;
@@ -459,7 +459,7 @@ define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32>
define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<165>;
@@ -515,7 +515,7 @@ define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32>
; elem_size=16, idx_size=4
define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -534,7 +534,7 @@ define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -551,7 +551,7 @@ define <8 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -569,7 +569,7 @@ define <16 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -589,7 +589,7 @@ define <32 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -614,7 +614,7 @@ define <64 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -649,7 +649,7 @@ define <128 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32>
define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<169>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
index 70e42c4d31df24..104184fbe130d9 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4
define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <1 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -26,7 +26,7 @@ define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -49,7 +49,7 @@ define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
define <32 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -67,7 +67,7 @@ define <32 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
define <64 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -87,7 +87,7 @@ define <64 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata
define <128 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<39>;
@@ -111,7 +111,7 @@ define <128 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %
define <256 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<77>;
@@ -144,7 +144,7 @@ define <256 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %
define <512 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<153>;
@@ -197,7 +197,7 @@ define <512 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %
; elem_size=16, idx_size=4
define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <1 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -216,7 +216,7 @@ define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
define <16 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -234,7 +234,7 @@ define <16 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metada
define <32 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -254,7 +254,7 @@ define <32 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
define <64 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<38>;
@@ -278,7 +278,7 @@ define <64 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metada
define <128 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<75>;
@@ -311,7 +311,7 @@ define <128 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32>
define <256 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<149>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
index ef31404cfbccd1..9e8c0175305d10 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -30,7 +30,7 @@ define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; elem_size=8, idx_size=4, rpf=2
define <32 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -49,7 +49,7 @@ define <32 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; elem_size=8, idx_size=4, rpf=4
define <64 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -70,7 +70,7 @@ define <64 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadat
; elem_size=8, idx_size=4, rpf=8
define <128 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<39>;
@@ -95,7 +95,7 @@ define <128 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %
; elem_size=8, idx_size=4, rpf=16
define <256 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<77>;
@@ -129,7 +129,7 @@ define <256 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32>
; elem_size=8, idx_size=4, rpf=32
define <512 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<153>;
@@ -182,7 +182,7 @@ define <512 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32>
; elem_size=16, idx_size=4, rpf=1
define <16 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<11>;
@@ -201,7 +201,7 @@ define <16 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metad
; elem_size=16, idx_size=4, rpf=2
define <32 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<20>;
@@ -222,7 +222,7 @@ define <32 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metad
; elem_size=16, idx_size=4, rpf=4
define <64 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<38>;
@@ -247,7 +247,7 @@ define <64 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metad
; elem_size=16, idx_size=4, rpf=8
define <128 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<75>;
@@ -281,7 +281,7 @@ define <128 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32>
; elem_size=16, idx_size=4, rpf=16
define <256 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<149>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
index 8370fe02adf01f..b1b95c6094ed0f 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=2, rpf=1
define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -30,7 +30,7 @@ define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata,
; elem_size=8, idx_size=2, rpf=2
define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -48,7 +48,7 @@ define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata,
; elem_size=8, idx_size=2, rpf=4
define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -66,7 +66,7 @@ define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata
; elem_size=8, idx_size=2, rpf=8
define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v16i8(i32 %metadata, <16 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -85,7 +85,7 @@ define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata
; elem_size=8, idx_size=2, rpf=16
define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v32i8(<2 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -107,7 +107,7 @@ define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %m
; elem_size=8, idx_size=2, rpf=32
define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v64i8(<4 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -135,7 +135,7 @@ define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %
; elem_size=8, idx_size=2, rpf=64
define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v128i8(<8 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v128i8(<8 x i32> %metadata, <128 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -176,7 +176,7 @@ define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %
; elem_size=8, idx_size=4, rpf=1
define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -200,7 +200,7 @@ define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
; elem_size=8, idx_size=4, rpf=2
define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -218,7 +218,7 @@ define <8 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata,
; elem_size=8, idx_size=4, rpf=4
define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -236,7 +236,7 @@ define <16 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; elem_size=8, idx_size=4, rpf=8
define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -255,7 +255,7 @@ define <32 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %me
; elem_size=8, idx_size=4, rpf=16
define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -277,7 +277,7 @@ define <64 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %m
; elem_size=8, idx_size=4, rpf=32
define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -306,7 +306,7 @@ define <128 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %
; elem_size=8, idx_size=4, rpf=64
define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<113>;
@@ -349,7 +349,7 @@ define <256 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32>
; elem_size=16, idx_size=2, rpf=1
define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -367,7 +367,7 @@ define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
; elem_size=16, idx_size=2, rpf=2
define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -385,7 +385,7 @@ define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadat
; elem_size=16, idx_size=2, rpf=4
define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -404,7 +404,7 @@ define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metada
; elem_size=16, idx_size=2, rpf=8
define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v16i16(i32 %metadata, <16 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<26>;
@@ -426,7 +426,7 @@ define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metada
; elem_size=16, idx_size=2, rpf=16
define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v32i16(<2 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<51>;
@@ -454,7 +454,7 @@ define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32>
; elem_size=16, idx_size=2, rpf=32
define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v64i16(<4 x i32> %metadata, <64 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<101>;
@@ -494,7 +494,7 @@ define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32>
; elem_size=16, idx_size=2, rpf=64
define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32> %metadata, <128 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v128i16(<8 x i32> %metadata, <128 x i16> %compdata, /* idx_size=.b2 */ i32 2, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v128i16(<8 x i32> %metadata, <128 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<201>;
@@ -559,7 +559,7 @@ define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32>
; elem_size=16, idx_size=4, rpf=1
define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -577,7 +577,7 @@ define <4 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; elem_size=16, idx_size=4, rpf=2
define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -595,7 +595,7 @@ define <8 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadat
; elem_size=16, idx_size=4, rpf=4
define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<14>;
@@ -614,7 +614,7 @@ define <16 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; elem_size=16, idx_size=4, rpf=8
define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<27>;
@@ -636,7 +636,7 @@ define <32 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %
; elem_size=16, idx_size=4, rpf=16
define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<53>;
@@ -664,7 +664,7 @@ define <64 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32>
; elem_size=16, idx_size=4, rpf=32
define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<105>;
@@ -705,7 +705,7 @@ define <128 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32>
; elem_size=16, idx_size=4, rpf=64
define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v16i32.v128i16(<16 x i32> %metadata, <128 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 4)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v16i32.v128i16(<16 x i32> %metadata, <128 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<209>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
index 9a58dee3ff4e75..3f11992d8269c7 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b16 %rs<3>;
@@ -30,7 +30,7 @@ define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
; elem_size=8, idx_size=4, rpf=2
define <16 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -48,7 +48,7 @@ define <16 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; elem_size=8, idx_size=4, rpf=4
define <32 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -67,7 +67,7 @@ define <32 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata
; elem_size=8, idx_size=4, rpf=8
define <64 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -88,7 +88,7 @@ define <64 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %me
; elem_size=8, idx_size=4, rpf=16
define <128 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -114,7 +114,7 @@ define <128 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %
; elem_size=8, idx_size=4, rpf=32
define <256 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -151,7 +151,7 @@ define <256 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %
; elem_size=8, idx_size=4, rpf=64
define <512 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<177>;
@@ -210,7 +210,7 @@ define <512 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32>
; elem_size=16, idx_size=4, rpf=1
define <8 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, <2 x i16> %compdata) {
; FORMAT-LABEL: define <8 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v2i16(i32 %metadata, <2 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -228,7 +228,7 @@ define <8 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; elem_size=16, idx_size=4, rpf=2
define <16 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <4 x i16> %compdata) {
; FORMAT-LABEL: define <16 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v4i16(i32 %metadata, <4 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -247,7 +247,7 @@ define <16 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metada
; elem_size=16, idx_size=4, rpf=4
define <32 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <8 x i16> %compdata) {
; FORMAT-LABEL: define <32 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <32 x i16> @llvm.nvvm.spdecompress.v32i16.i32.v8i16(i32 %metadata, <8 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<22>;
@@ -268,7 +268,7 @@ define <32 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metada
; elem_size=16, idx_size=4, rpf=8
define <64 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <16 x i16> %compdata) {
; FORMAT-LABEL: define <64 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <64 x i16> @llvm.nvvm.spdecompress.v64i16.v2i32.v16i16(<2 x i32> %metadata, <16 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<43>;
@@ -294,7 +294,7 @@ define <64 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %
; elem_size=16, idx_size=4, rpf=16
define <128 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <32 x i16> %compdata) {
; FORMAT-LABEL: define <128 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <128 x i16> @llvm.nvvm.spdecompress.v128i16.v4i32.v32i16(<4 x i32> %metadata, <32 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<85>;
@@ -330,7 +330,7 @@ define <128 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32>
; elem_size=16, idx_size=4, rpf=32
define <256 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <64 x i16> %compdata) {
; FORMAT-LABEL: define <256 x i16> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <256 x i16> @llvm.nvvm.spdecompress.v256i16.v8i32.v64i16(<8 x i32> %metadata, <64 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<169>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
index d1269a0cf064d1..88d97483775b87 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<7>;
@@ -24,7 +24,7 @@ define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; elem_size=8, idx_size=4, rpf=2
define <32 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<12>;
@@ -43,7 +43,7 @@ define <32 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; elem_size=8, idx_size=4, rpf=4
define <64 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<23>;
@@ -64,7 +64,7 @@ define <64 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %m
; elem_size=8, idx_size=4, rpf=8
define <128 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<45>;
@@ -90,7 +90,7 @@ define <128 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %
; elem_size=8, idx_size=4, rpf=16
define <256 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<89>;
@@ -127,7 +127,7 @@ define <256 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32>
; elem_size=8, idx_size=4, rpf=32
define <512 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 16)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<177>;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
index 30fba70f9989cc..d670be61271390 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
@@ -6,7 +6,7 @@
; elem_size=8, idx_size=4, rpf=1
define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
; FORMAT-LABEL: define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
-; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v4i8(i32 %metadata, <4 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<5>;
@@ -24,7 +24,7 @@ define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
; elem_size=8, idx_size=4, rpf=2
define <16 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <8 x i8> %compdata) {
; FORMAT-LABEL: define <16 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
-; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v8i8(i32 %metadata, <8 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<8>;
@@ -42,7 +42,7 @@ define <16 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; elem_size=8, idx_size=4, rpf=4
define <32 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <16 x i8> %compdata) {
; FORMAT-LABEL: define <32 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
-; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.v2i32.v16i8(<2 x i32> %metadata, <16 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<15>;
@@ -61,7 +61,7 @@ define <32 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %me
; elem_size=8, idx_size=4, rpf=8
define <64 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <32 x i8> %compdata) {
; FORMAT-LABEL: define <64 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
-; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <64 x i8> @llvm.nvvm.spdecompress.v64i8.v4i32.v32i8(<4 x i32> %metadata, <32 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<29>;
@@ -83,7 +83,7 @@ define <64 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %me
; elem_size=8, idx_size=4, rpf=16
define <128 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <64 x i8> %compdata) {
; FORMAT-LABEL: define <128 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
-; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <128 x i8> @llvm.nvvm.spdecompress.v128i8.v8i32.v64i8(<8 x i32> %metadata, <64 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<57>;
@@ -112,7 +112,7 @@ define <128 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %
; elem_size=8, idx_size=4, rpf=32
define <256 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <128 x i8> %compdata) {
; FORMAT-LABEL: define <256 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
-; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <256 x i8> @llvm.nvvm.spdecompress.v256i8.v16i32.v128i8(<16 x i32> %metadata, <128 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<113>;
@@ -155,7 +155,7 @@ define <256 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32>
; elem_size=8, idx_size=4, rpf=64
define <512 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32> %metadata, <256 x i8> %compdata) {
; FORMAT-LABEL: define <512 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
-; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v32i32.v256i8(<32 x i32> %metadata, <256 x i8> %compdata, /* idx_size=.b4 */ i32 4, /* num_tgt= */ i32 8)
+; FORMAT-NEXT: %res = call <512 x i8> @llvm.nvvm.spdecompress.v512i8.v32i32.v256i8(<32 x i32> %metadata, <256 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
; CHECK: {
; CHECK-NEXT: .reg .b32 %r<225>;
diff --git a/llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn b/llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn
index 1135884bdfd68e..43b5efee01185a 100644
--- a/llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn
+++ b/llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn
@@ -96,5 +96,6 @@ static_library("IR") {
"VectorTypeUtils.cpp",
"Verifier.cpp",
"VerifierAMDGPU.cpp",
+ "VerifierNVPTX.cpp",
]
}
>From 9a0a78993b511d5d19494349ec63592001cd1f51 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Wed, 16 Sep 2026 18:14:32 +0200
Subject: [PATCH 06/10] [NVPTX] Avoid redundant packing for partial SP operands
---
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp | 10 ++++
.../CodeGen/NVPTX/spdecompress-sp1to16.ll | 31 ++++------
.../test/CodeGen/NVPTX/spdecompress-sp1to2.ll | 36 ++++--------
.../test/CodeGen/NVPTX/spdecompress-sp1to4.ll | 58 ++++++-------------
.../test/CodeGen/NVPTX/spdecompress-sp1to8.ll | 29 +++-------
.../CodeGen/NVPTX/spdecompress-sp2to16.ll | 14 ++---
.../test/CodeGen/NVPTX/spdecompress-sp2to4.ll | 28 +++------
.../test/CodeGen/NVPTX/spdecompress-sp2to8.ll | 14 ++---
8 files changed, 73 insertions(+), 147 deletions(-)
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
index 0a9ea79315d882..b9290bf79effbf 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
@@ -3281,6 +3281,16 @@ static void appendSplitSPOperands(ArrayRef<SDUse> Input,
continue;
}
+ EVT VT = Operand.getValueType();
+ // Pack partial .b32 operands at their natural width before extending them
+ // to avoid redundant cvt/prmt sequences.
+ if (VT.getSizeInBits() < 32) {
+ EVT IntVT = EVT::getIntegerVT(*DAG.getContext(), VT.getSizeInBits());
+ SDValue Packed = DAG.getBitcast(IntVT, Operand);
+ Output.push_back(DAG.getAnyExtOrTrunc(Packed, SDLoc(Operand), MVT::i32));
+ continue;
+ }
+
SmallVector<SDValue> Parts;
splitSPVector(Operand, Parts, DAG);
for (SDValue Part : Parts)
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
index e4a1302cf2c749..ca0abdd84e92f5 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
@@ -9,16 +9,13 @@ define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<13>;
+; CHECK-NEXT: .reg .b32 %r<7>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b8 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: prmt.b32 %r2, %r1, %r3, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r4, %r5, %r6, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r7, %r2, %r4, 0x5410U;
-; CHECK-NEXT: ld.param::func.b32 %r8, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x1 {%r9, %r10, %r11, %r12}, {%r8}, {%r7};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r9, %r10, %r11, %r12};
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6}, {%r2}, {%r1};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
%res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, i32 4, i32 16)
ret <16 x i8> %res
@@ -29,20 +26,14 @@ define <32 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadat
; FORMAT-NEXT: %res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<17>;
+; CHECK-NEXT: .reg .b32 %r<11>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x2 {%r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r13, %r14, %r15, %r16};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r9, %r10, %r11, %r12};
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
%res = call <32 x i8> @llvm.nvvm.spdecompress.v32i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 16)
ret <32 x i8> %res
@@ -179,12 +170,10 @@ define <16 x i16> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metad
; FORMAT-NEXT: %res = call <16 x i16> @llvm.nvvm.spdecompress.v16i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<11>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b16 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_1];
; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_0];
; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
index 65f5a6e3074b02..be1f3111e599c0 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
@@ -8,19 +8,13 @@ define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata,
; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<10>;
+; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x2 {%r9}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x2 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
%res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 2)
ret <4 x i8> %res
@@ -131,19 +125,13 @@ define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata,
; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<10>;
+; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x2 {%r9}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x2 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
%res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 2)
ret <4 x i8> %res
@@ -255,12 +243,10 @@ define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_1];
-; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b16 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_1];
; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_0];
; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x1 {%r3}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
@@ -413,12 +399,10 @@ define <2 x i16> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; FORMAT-NEXT: %res = call <2 x i16> @llvm.nvvm.spdecompress.v2i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 2)
; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b16 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_1];
; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_0];
; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x1 {%r3}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
index 926f2186d4649d..28c5a9a4a67fd6 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
@@ -9,16 +9,13 @@ define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata,
; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<10>;
+; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b8 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
-; CHECK-NEXT: prmt.b32 %r2, %r1, %r3, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r4, %r5, %r6, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r7, %r2, %r4, 0x5410U;
-; CHECK-NEXT: ld.param::func.b32 %r8, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
-; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x1 {%r9}, {%r8}, {%r7};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x1 {%r3}, {%r2}, {%r1};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
%res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, i32 2, i32 4)
ret <4 x i8> %res
@@ -29,19 +26,13 @@ define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata,
; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-NEXT: .reg .b32 %r<5>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x2 {%r9, %r10}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r9, %r10};
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
%res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 4)
ret <8 x i8> %res
@@ -168,16 +159,13 @@ define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<10>;
+; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b8 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: prmt.b32 %r2, %r1, %r3, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r4, %r5, %r6, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r7, %r2, %r4, 0x5410U;
-; CHECK-NEXT: ld.param::func.b32 %r8, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x1 {%r9}, {%r8}, {%r7};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x1 {%r3}, {%r2}, {%r1};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
%res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, i32 4, i32 4)
ret <4 x i8> %res
@@ -188,19 +176,13 @@ define <8 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata,
; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-NEXT: .reg .b32 %r<5>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x2 {%r9, %r10}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r9, %r10};
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
%res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 4)
ret <8 x i8> %res
@@ -328,12 +310,10 @@ define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadat
; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<5>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_1];
-; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b16 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_1];
; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_0];
; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x1 {%r3, %r4}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
@@ -518,12 +498,10 @@ define <4 x i16> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; FORMAT-NEXT: %res = call <4 x i16> @llvm.nvvm.spdecompress.v4i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<5>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b16 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_1];
; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_0];
; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x1 {%r3, %r4}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
index 104184fbe130d9..f39fe228079c35 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
@@ -9,16 +9,13 @@ define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-NEXT: .reg .b32 %r<5>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b8 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: prmt.b32 %r2, %r1, %r3, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r4, %r5, %r6, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r7, %r2, %r4, 0x5410U;
-; CHECK-NEXT: ld.param::func.b32 %r8, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x1 {%r9, %r10}, {%r8}, {%r7};
-; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r9, %r10};
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x1 {%r3, %r4}, {%r2}, {%r1};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
%res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v1i8(i32 %metadata, <1 x i8> %compdata, i32 4, i32 8)
ret <8 x i8> %res
@@ -29,19 +26,13 @@ define <16 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata
; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<13>;
+; CHECK-NEXT: .reg .b32 %r<7>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x2 {%r9, %r10, %r11, %r12}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r9, %r10, %r11, %r12};
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
%res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 8)
ret <16 x i8> %res
@@ -200,12 +191,10 @@ define <8 x i16> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadat
; FORMAT-NEXT: %res = call <8 x i16> @llvm.nvvm.spdecompress.v8i16.i32.v1i16(i32 %metadata, <1 x i16> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
; CHECK-NEXT: .reg .b32 %r<7>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
-; CHECK-NEXT: ld.param::func.b16 %rs1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: mov.b32 %r1, {%rs1, %rs2};
+; CHECK-NEXT: ld.param::func.b16 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_1];
; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_0];
; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x1 {%r3, %r4, %r5, %r6}, {%r2}, {%r1};
; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
index 9e8c0175305d10..07d23531be194d 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
@@ -9,19 +9,13 @@ define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadat
; FORMAT-NEXT: %res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 16)
; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<13>;
+; CHECK-NEXT: .reg .b32 %r<7>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x1 {%r9, %r10, %r11, %r12}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r9, %r10, %r11, %r12};
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
; CHECK-NEXT: ret;
%res = call <16 x i8> @llvm.nvvm.spdecompress.v16i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 16)
ret <16 x i8> %res
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
index b1b95c6094ed0f..9b2e299ac61efa 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
@@ -9,19 +9,13 @@ define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata,
; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 2, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<10>;
+; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x1 {%r9}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
%res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 2, i32 4)
ret <4 x i8> %res
@@ -179,19 +173,13 @@ define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
; FORMAT-NEXT: %res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 4)
; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<10>;
+; CHECK-NEXT: .reg .b32 %r<4>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x1 {%r9}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.b32 [func_retval0], %r9;
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
; CHECK-NEXT: ret;
%res = call <4 x i8> @llvm.nvvm.spdecompress.v4i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 4)
ret <4 x i8> %res
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
index 3f11992d8269c7..0fb236306daad3 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
@@ -9,19 +9,13 @@ define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata,
; FORMAT-NEXT: %res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, /* idx_size= */ i32 4, /* num_tgt= */ i32 8)
; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
; CHECK: {
-; CHECK-NEXT: .reg .b16 %rs<3>;
-; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-NEXT: .reg .b32 %r<5>;
; CHECK-EMPTY:
; CHECK-NEXT: // %bb.0:
; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
-; CHECK-NEXT: ld.param::func.v2.b8 {%rs1, %rs2}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
-; CHECK-NEXT: cvt.u32.u16 %r2, %rs2;
-; CHECK-NEXT: cvt.u32.u16 %r3, %rs1;
-; CHECK-NEXT: prmt.b32 %r4, %r3, %r2, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r5, %r6, %r7, 0x3340U;
-; CHECK-NEXT: prmt.b32 %r8, %r4, %r5, 0x5410U;
-; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x1 {%r9, %r10}, {%r1}, {%r8};
-; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r9, %r10};
+; CHECK-NEXT: ld.param::func.b16 %r2, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
; CHECK-NEXT: ret;
%res = call <8 x i8> @llvm.nvvm.spdecompress.v8i8.i32.v2i8(i32 %metadata, <2 x i8> %compdata, i32 4, i32 8)
ret <8 x i8> %res
>From b2c8ab96c1da7c060a8f9c323ff3daabff68a872 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Fri, 18 Sep 2026 09:25:03 +0200
Subject: [PATCH 07/10] [NVPTX] Add -mattr=+ptx94 to the sp{de}compress tests
---
llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll | 4 ++--
llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll | 4 ++--
10 files changed, 20 insertions(+), 20 deletions(-)
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
index 6dd381d767b2b0..9c3f3bad6e98ed 100644
--- a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
define {i32, <4 x i8>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <8 x i8> %data) {
; FORMAT-LABEL: define { i32, <4 x i8> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
index ca0abdd84e92f5..c146017fc8b4ac 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4
define <16 x i8> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <1 x i8> %compdata) {
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
index be1f3111e599c0..f4d55919f7fde8 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, <2 x i8> %compdata) {
; FORMAT-LABEL: define <4 x i8> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
index 28c5a9a4a67fd6..69b5d30a110877 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=2
define <4 x i8> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <1 x i8> %compdata) {
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
index f39fe228079c35..1b246478fe1f74 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4
define <8 x i8> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <1 x i8> %compdata) {
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
index 07d23531be194d..9e88aa631afc42 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
define <16 x i8> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <2 x i8> %compdata) {
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
index 9b2e299ac61efa..883a0abe7f487e 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=2, rpf=1
define <4 x i8> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, <2 x i8> %compdata) {
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
index 0fb236306daad3..73dcbf898f3ee3 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
define <8 x i8> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <2 x i8> %compdata) {
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
index 88d97483775b87..743c66e9350bbe 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
define <16 x i8> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
index d670be61271390..614ab191b46db6 100644
--- a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
@@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | FileCheck %s
; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
-; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+; RUN: %if ptxas-sm_107a && ptxas-isa-9.4 %{ llc < %s -march=nvptx64 -mcpu=sm_107a -mattr=+ptx94 | %ptxas-verify -arch=sm_107a %}
; elem_size=8, idx_size=4, rpf=1
define <8 x i8> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, <4 x i8> %compdata) {
>From 25ba1c83ba15f6d5b683b42bcea1feb79efd2605 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Fri, 18 Sep 2026 16:28:52 +0200
Subject: [PATCH 08/10] [IR] Rename the NVPTX intrinsic verifier to NVVM
---
llvm/lib/IR/CMakeLists.txt | 2 +-
llvm/lib/IR/Verifier.cpp | 2 +-
llvm/lib/IR/VerifierInternal.h | 6 +++---
llvm/lib/IR/{VerifierNVPTX.cpp => VerifierNVVM.cpp} | 6 +++---
llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn | 2 +-
5 files changed, 9 insertions(+), 9 deletions(-)
rename llvm/lib/IR/{VerifierNVPTX.cpp => VerifierNVVM.cpp} (96%)
diff --git a/llvm/lib/IR/CMakeLists.txt b/llvm/lib/IR/CMakeLists.txt
index 24dc690ae97594..8ce5d4e6c7b895 100644
--- a/llvm/lib/IR/CMakeLists.txt
+++ b/llvm/lib/IR/CMakeLists.txt
@@ -80,7 +80,7 @@ add_llvm_component_library(LLVMCore
VectorTypeUtils.cpp
Verifier.cpp
VerifierAMDGPU.cpp
- VerifierNVPTX.cpp
+ VerifierNVVM.cpp
VFABIDemangler.cpp
RuntimeLibcalls.cpp
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 25f1f8b6437659..42a803aa362280 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -7415,7 +7415,7 @@ void Verifier::visitIntrinsicCall(Intrinsic::ID ID, CallBase &Call) {
// Target-specific intrinsic call checks.
verifyAMDGPUIntrinsicCall(*this, ID, Call);
- verifyNVPTXIntrinsicCall(*this, ID, Call);
+ verifyNVVMIntrinsicCall(*this, ID, Call);
}
/// Carefully grab the subprogram from a local scope.
diff --git a/llvm/lib/IR/VerifierInternal.h b/llvm/lib/IR/VerifierInternal.h
index f38947de2f0bb5..c7dbb9b5a6c91f 100644
--- a/llvm/lib/IR/VerifierInternal.h
+++ b/llvm/lib/IR/VerifierInternal.h
@@ -230,10 +230,10 @@ bool isAMDGPUCallBrIntrinsic(Intrinsic::ID ID);
//==============================================================================
-// NVPTX-specific verification functions
+// NVVM-specific verification functions
-void verifyNVPTXIntrinsicCall(VerifierSupport &VS, Intrinsic::ID ID,
- CallBase &Call);
+void verifyNVVMIntrinsicCall(VerifierSupport &VS, Intrinsic::ID ID,
+ CallBase &Call);
//==============================================================================
diff --git a/llvm/lib/IR/VerifierNVPTX.cpp b/llvm/lib/IR/VerifierNVVM.cpp
similarity index 96%
rename from llvm/lib/IR/VerifierNVPTX.cpp
rename to llvm/lib/IR/VerifierNVVM.cpp
index e0e1ec292be709..3393ce1a888b25 100644
--- a/llvm/lib/IR/VerifierNVPTX.cpp
+++ b/llvm/lib/IR/VerifierNVVM.cpp
@@ -7,7 +7,7 @@
//===----------------------------------------------------------------------===//
///
/// \file
-/// This file contains NVPTX-specific IR verification logic. These checks are
+/// This file contains NVVM-specific IR verification logic. These checks are
/// always compiled and linked as part of LLVMCore.
///
//===----------------------------------------------------------------------===//
@@ -116,8 +116,8 @@ static void verifySPDecompress(VerifierSupport &VS, CallBase &Call) {
"invalid llvm.nvvm.spdecompress layout", &Call);
}
-void llvm::verifyNVPTXIntrinsicCall(VerifierSupport &VS, Intrinsic::ID ID,
- CallBase &Call) {
+void llvm::verifyNVVMIntrinsicCall(VerifierSupport &VS, Intrinsic::ID ID,
+ CallBase &Call) {
switch (ID) {
default:
return;
diff --git a/llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn b/llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn
index 43b5efee01185a..2ec04a513c0539 100644
--- a/llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn
+++ b/llvm/utils/gn/secondary/llvm/lib/IR/BUILD.gn
@@ -96,6 +96,6 @@ static_library("IR") {
"VectorTypeUtils.cpp",
"Verifier.cpp",
"VerifierAMDGPU.cpp",
- "VerifierNVPTX.cpp",
+ "VerifierNVVM.cpp",
]
}
>From e11ba554671a671bf2a9dc829981d9691a856478 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Sun, 20 Sep 2026 21:19:44 +0200
Subject: [PATCH 09/10] [NVPTX] Move sp{de}compress instruction selection to
TableGen
---
.../NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp | 4 +-
.../NVPTX/MCTargetDesc/NVPTXInstPrinter.h | 2 +-
llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp | 67 ------
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp | 74 ++++++-
llvm/lib/Target/NVPTX/NVPTXIntrinsics.td | 200 ++++++++++++++----
.../Target/NVPTX/NVPTXSelectionDAGInfo.cpp | 6 -
llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h | 2 -
7 files changed, 229 insertions(+), 126 deletions(-)
diff --git a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
index 5274f1cacad4d6..a890e577b6cd66 100644
--- a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
+++ b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
@@ -486,7 +486,7 @@ static void printSPRegList(NVPTXInstPrinter *Printer, const MCInst *MI,
O << "}";
}
-void NVPTXInstPrinter::printSPQualifier(const MCInst *MI, int OpNum,
+void NVPTXInstPrinter::printSPModifiers(const MCInst *MI, int OpNum,
const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier) {
if (Modifier.empty()) {
@@ -540,7 +540,7 @@ void NVPTXInstPrinter::printSPQualifier(const MCInst *MI, int OpNum,
auto Layout = GetSPLayout(nvvm::getSPCompressLayout);
assert(Layout && "invalid spcompress qualifiers");
- unsigned FirstVariableOp = OpNum + 3;
+ unsigned FirstVariableOp = OpNum + 5;
assert(static_cast<unsigned>(OpNum) ==
Layout->MetadataSize + Layout->CompressedDataSize &&
MI->getNumOperands() == FirstVariableOp + Layout->DataSize + 1 &&
diff --git a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
index dd8e7b499e3f9e..b19e9f73cbe0f5 100644
--- a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
+++ b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
@@ -53,7 +53,7 @@ class NVPTXInstPrinter : public MCInstPrinter {
raw_ostream &O);
void printMmaCode(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier = {});
- void printSPQualifier(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
+ void printSPModifiers(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier = {});
void printMemOperand(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier = {});
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
index 68d262bf125329..81badbc1c3b101 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
@@ -138,7 +138,6 @@ class NVPTXDAGToDAGISel : public SelectionDAGISel {
bool IsIm2Col = false);
void SelectTcgen05Ld(SDNode *N, bool hasOffset = false);
void SelectTcgen05St(SDNode *N, bool hasOffset = false);
- void selectSPNode(SDNode *N);
void selectAtomicSwap128(SDNode *N);
inline SDValue getI32Imm(unsigned Imm, const SDLoc &DL) {
@@ -290,10 +289,6 @@ void NVPTXDAGToDAGISel::Select(SDNode *N) {
if (tryStoreVector(N))
return;
break;
- case NVPTXISD::SPCOMPRESS:
- case NVPTXISD::SPDECOMPRESS:
- selectSPNode(N);
- return;
case ISD::INTRINSIC_W_CHAIN:
if (tryIntrinsicChain(N))
return;
@@ -446,68 +441,6 @@ void NVPTXDAGToDAGISel::SelectTcgen05Ld(SDNode *N, bool hasOffset) {
}
}
-// TableGen's variadic morph encoding supports at most seven fixed operands,
-// while SP register bundles can be much wider, so select their opcodes here.
-static unsigned getSPCompressOpcode(unsigned NumResults) {
-#define SPCOMPRESS_OPCODE_CASE(NumResults) \
- case NumResults: \
- return NVPTX::SPCOMPRESS_OUT_##NumResults
-
- switch (NumResults) {
- SPCOMPRESS_OPCODE_CASE(2);
- SPCOMPRESS_OPCODE_CASE(3);
- SPCOMPRESS_OPCODE_CASE(5);
- SPCOMPRESS_OPCODE_CASE(6);
- SPCOMPRESS_OPCODE_CASE(9);
- SPCOMPRESS_OPCODE_CASE(10);
- SPCOMPRESS_OPCODE_CASE(12);
- SPCOMPRESS_OPCODE_CASE(18);
- SPCOMPRESS_OPCODE_CASE(20);
- SPCOMPRESS_OPCODE_CASE(24);
- SPCOMPRESS_OPCODE_CASE(36);
- SPCOMPRESS_OPCODE_CASE(40);
- SPCOMPRESS_OPCODE_CASE(48);
- SPCOMPRESS_OPCODE_CASE(72);
- SPCOMPRESS_OPCODE_CASE(80);
- SPCOMPRESS_OPCODE_CASE(96);
- default:
- llvm_unreachable("invalid spcompress result count");
- }
-
-#undef SPCOMPRESS_OPCODE_CASE
-}
-
-static unsigned getSPDecompressOpcode(unsigned NumResults) {
-#define SPDECOMPRESS_OPCODE_CASE(NumResults) \
- case NumResults: \
- return NVPTX::SPDECOMPRESS_OUT_##NumResults
-
- switch (NumResults) {
- SPDECOMPRESS_OPCODE_CASE(1);
- SPDECOMPRESS_OPCODE_CASE(2);
- SPDECOMPRESS_OPCODE_CASE(4);
- SPDECOMPRESS_OPCODE_CASE(8);
- SPDECOMPRESS_OPCODE_CASE(16);
- SPDECOMPRESS_OPCODE_CASE(32);
- SPDECOMPRESS_OPCODE_CASE(64);
- SPDECOMPRESS_OPCODE_CASE(128);
- default:
- llvm_unreachable("invalid spdecompress result count");
- }
-
-#undef SPDECOMPRESS_OPCODE_CASE
-}
-
-void NVPTXDAGToDAGISel::selectSPNode(SDNode *N) {
- unsigned Opcode = N->getOpcode() == NVPTXISD::SPCOMPRESS
- ? getSPCompressOpcode(N->getNumValues())
- : getSPDecompressOpcode(N->getNumValues());
- SmallVector<SDValue, 16> Operands;
- llvm::append_range(Operands, N->ops());
- ReplaceNode(
- N, CurDAG->getMachineNode(Opcode, SDLoc(N), N->getVTList(), Operands));
-}
-
bool NVPTXDAGToDAGISel::tryIntrinsicChain(SDNode *N) {
unsigned IID = N->getConstantOperandVal(1);
switch (IID) {
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
index b9290bf79effbf..dc10190c287805 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
@@ -3300,6 +3300,58 @@ static void appendSplitSPOperands(ArrayRef<SDUse> Input,
}
}
+static unsigned getSPCompressNodeOpcode(unsigned NumResults) {
+ // SelectionDAG node descriptions have a fixed result arity, so use one node
+ // kind for each result-register count selected by the legal SP layouts.
+#define SPCOMPRESS_NODE_CASE(NumResults) \
+ case NumResults: \
+ return NVPTXISD::SPCOMPRESS_R##NumResults
+
+ switch (NumResults) {
+ SPCOMPRESS_NODE_CASE(2);
+ SPCOMPRESS_NODE_CASE(3);
+ SPCOMPRESS_NODE_CASE(5);
+ SPCOMPRESS_NODE_CASE(6);
+ SPCOMPRESS_NODE_CASE(9);
+ SPCOMPRESS_NODE_CASE(10);
+ SPCOMPRESS_NODE_CASE(12);
+ SPCOMPRESS_NODE_CASE(18);
+ SPCOMPRESS_NODE_CASE(20);
+ SPCOMPRESS_NODE_CASE(24);
+ SPCOMPRESS_NODE_CASE(36);
+ SPCOMPRESS_NODE_CASE(40);
+ SPCOMPRESS_NODE_CASE(48);
+ SPCOMPRESS_NODE_CASE(72);
+ SPCOMPRESS_NODE_CASE(80);
+ SPCOMPRESS_NODE_CASE(96);
+ default:
+ llvm_unreachable("invalid spcompress result count");
+ }
+
+#undef SPCOMPRESS_NODE_CASE
+}
+
+static unsigned getSPDecompressNodeOpcode(unsigned NumResults) {
+#define SPDECOMPRESS_NODE_CASE(NumResults) \
+ case NumResults: \
+ return NVPTXISD::SPDECOMPRESS_R##NumResults
+
+ switch (NumResults) {
+ SPDECOMPRESS_NODE_CASE(1);
+ SPDECOMPRESS_NODE_CASE(2);
+ SPDECOMPRESS_NODE_CASE(4);
+ SPDECOMPRESS_NODE_CASE(8);
+ SPDECOMPRESS_NODE_CASE(16);
+ SPDECOMPRESS_NODE_CASE(32);
+ SPDECOMPRESS_NODE_CASE(64);
+ SPDECOMPRESS_NODE_CASE(128);
+ default:
+ llvm_unreachable("invalid spdecompress result count");
+ }
+
+#undef SPDECOMPRESS_NODE_CASE
+}
+
static SDValue lowerSPIntrinsic(SDValue Op, ArrayRef<SDValue> Qualifiers,
SelectionDAG &DAG) {
SDNode *N = Op.getNode();
@@ -3321,8 +3373,9 @@ static SDValue lowerSPIntrinsic(SDValue Op, ArrayRef<SDValue> Qualifiers,
assert((IID == Intrinsic::nvvm_spcompress ||
IID == Intrinsic::nvvm_spdecompress) &&
"unexpected SP intrinsic");
- unsigned Opcode = IID == Intrinsic::nvvm_spcompress ? NVPTXISD::SPCOMPRESS
- : NVPTXISD::SPDECOMPRESS;
+ unsigned Opcode = IID == Intrinsic::nvvm_spcompress
+ ? getSPCompressNodeOpcode(ResultTys.size())
+ : getSPDecompressNodeOpcode(ResultTys.size());
SDValue Lowered = DAG.getNode(Opcode, DL, ResultTys, Ops);
SmallVector<SDValue, 2> Retvals;
@@ -3347,18 +3400,31 @@ static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
SDNode *N = Op.getNode();
SDLoc DL(N);
EVT DataVT = N->getOperand(1).getValueType();
+ EVT CDataVT = N->getValueType(1);
+ assert(DataVT.isVector() && CDataVT.isVector() &&
+ "invalid spcompress types");
+
unsigned ElemSize = DataVT.getScalarSizeInBits();
unsigned IdxSize = cast<ConstantSDNode>(N->getOperand(3))->getZExtValue();
+ unsigned NumTgt = cast<ConstantSDNode>(N->getOperand(4))->getZExtValue();
+ unsigned DataElts = DataVT.getVectorNumElements();
+ unsigned NumSrcNumerator = NumTgt * CDataVT.getVectorNumElements();
+ assert(DataElts != 0 && NumSrcNumerator % DataElts == 0 &&
+ "invalid spcompress factor");
+ unsigned NumSrc = NumSrcNumerator / DataElts;
+
unsigned DataSize = getSPVectorNumParts(DataVT);
assert(DataSize % 2 == 0 && "invalid spcompress data size");
unsigned RepeatFactor = DataSize / 2;
assert(nvvm::isValidSPRepeatFactor(RepeatFactor) &&
"invalid spcompress repeat factor");
- SmallVector<SDValue, 3> Qualifiers = {
+ SmallVector<SDValue, 5> Qualifiers = {
DAG.getTargetConstant(ElemSize, DL, MVT::i32),
DAG.getTargetConstant(IdxSize, DL, MVT::i32),
- DAG.getTargetConstant(RepeatFactor, DL, MVT::i32)};
+ DAG.getTargetConstant(RepeatFactor, DL, MVT::i32),
+ DAG.getTargetConstant(NumSrc, DL, MVT::i32),
+ DAG.getTargetConstant(NumTgt, DL, MVT::i32)};
return lowerSPIntrinsic(Op, Qualifiers, DAG);
}
diff --git a/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td b/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
index 4a8361d3036693..dfa9dbd5d5f9fa 100644
--- a/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
+++ b/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
@@ -7697,71 +7697,183 @@ foreach space = ["tensor", "shared"] in {
} // cta_group
} // space
-def SPQualifier : Operand<i32> {
- let PrintMethod = "printSPQualifier";
+def SPModifiers : Operand<i32> {
+ let PrintMethod = "printSPModifiers";
}
class SPRegList<int NumRegs, string Prefix> {
- dag Dag = !dag(outs, !listsplat(B32, NumRegs),
- !foreach(I, !range(NumRegs), Prefix # I));
+ list<string> Names = !foreach(I, !range(NumRegs), Prefix # I);
+ dag Outs = !dag(outs, !listsplat(B32, NumRegs), Names);
+ dag Ins = !dag(ins, !listsplat(B32, NumRegs), Names);
+ dag Set = !dag(set, !listsplat(B32, NumRegs), Names);
+}
+
+class SPLayout<int ResultRegs, int InputRegs> {
+ int NumResultRegs = ResultRegs;
+ int NumInputRegs = InputRegs;
+}
+
+class SPCeilDiv<int Numerator, int Denominator> {
+ int Value = !div(!add(Numerator, !sub(Denominator, 1)), Denominator);
+}
+
+// The SelectionDAG nodes have varying result and operand counts. Give each
+// legal layout a fixed profile so instruction selection does not need a
+// variadic machine instruction.
+class SPTypeProfile<SPLayout Layout, int NumQualifiers>
+ : SDTypeProfile<
+ Layout.NumResultRegs, -1,
+ !foreach(I,
+ !range(!add(Layout.NumResultRegs, NumQualifiers,
+ Layout.NumInputRegs)),
+ SDTCisVT<I, i32>)>;
+
+class SPNode<string Opcode, SPLayout Layout, int NumQualifiers>
+ : SDNode<Opcode, SPTypeProfile<Layout, NumQualifiers>>;
+
+class SPExactMatch<dag Operands, dag Fragment, int NumOperands>
+ : PatFrag<Operands, Fragment,
+ [{ return N->getNumOperands() == }]
+ # !cast<string>(NumOperands) # [{; }]>;
+
+defvar SP_ELEM_SIZES = [8, 16];
+defvar SP_IDX_SIZES = [2, 4];
+defvar SP_REPEAT_FACTORS = [1, 2, 4, 8, 16, 32, 64];
+
+// mdata + cdata = ceil(repeat_factor * idx_size / elem_size) + repeat_factor;
+// spdesc + data = 1 + 2 * repeat_factor.
+defset list<SPLayout> SPCOMPRESS_LAYOUT_SET = {
+ foreach ElemSize = SP_ELEM_SIZES in
+ foreach IdxSize = SP_IDX_SIZES in
+ foreach RepeatFactor = SP_REPEAT_FACTORS in {
+ defvar ResultRegs =
+ !add(SPCeilDiv<!mul(RepeatFactor, IdxSize), ElemSize>.Value,
+ RepeatFactor);
+ defvar InputRegs = !add(1, !mul(2, RepeatFactor));
+ defvar Name = "SPCompressLayout_" # ResultRegs # _ # InputRegs;
+ if !not(!exists<SPLayout>(Name)) then
+ def Name : SPLayout<ResultRegs, InputRegs>;
+ }
}
-class SPFactorName<int NumSrc, int NumTgt> {
- string ret = ".sp::" # NumSrc # ":" # NumTgt;
+defvar SP_NUM_SRC = [1, 2, 4];
+defvar SP_NUM_TGT = [2, 4, 8, 16];
+
+// data = ceil(num_tgt * elem_size * repeat_factor / 32);
+// mdata = ceil(num_src * idx_size * repeat_factor / 32);
+// cdata = ceil(num_src * elem_size * repeat_factor / 32).
+defset list<SPLayout> SPDECOMPRESS_LAYOUT_SET = {
+ foreach NumSrc = SP_NUM_SRC in
+ foreach NumTgt = SP_NUM_TGT in
+ if !lt(NumSrc, NumTgt) then
+ foreach ElemSize = SP_ELEM_SIZES in
+ foreach IdxSize = SP_IDX_SIZES in
+ foreach RepeatFactor = SP_REPEAT_FACTORS in {
+ defvar DataBits = !mul(NumTgt, ElemSize, RepeatFactor);
+ if !and(!le(!mul(NumSrc, ElemSize), 32),
+ !or(!ne(IdxSize, 2), !le(NumTgt, 4)),
+ !ge(DataBits, 32), !le(DataBits, 4096)) then {
+ defvar ResultRegs = SPCeilDiv<DataBits, 32>.Value;
+ defvar InputRegs =
+ !add(SPCeilDiv<!mul(NumSrc, IdxSize, RepeatFactor),
+ 32>.Value,
+ SPCeilDiv<!mul(NumSrc, ElemSize, RepeatFactor),
+ 32>.Value);
+ defvar Name =
+ "SPDecompressLayout_" # ResultRegs # _ # InputRegs;
+ if !not(!exists<SPLayout>(Name)) then
+ def Name : SPLayout<ResultRegs, InputRegs>;
+ }
+ }
}
-// mdata + cdata = ceil(repeat_factor * idx_size / elem_size) + repeat_factor.
-// The first group is shared by .b8.b2 and .b16.b4. The next groups are the
-// additional unique counts for .b16.b2 and .b8.b4.
-defvar SPCOMPRESS_OPCODE_CASES =
- !sort(NumResultRegs,
- !listconcat([2, 3, 5, 10, 20, 40, 80], [9, 18, 36, 72],
- [6, 12, 24, 48, 96]),
- NumResultRegs);
-
-// data = ceil(num_tgt * elem_size * repeat_factor / 32). The legal
-// spdecompress layouts produce all power-of-two data register counts from 1 to
-// 128.
-defvar SPDECOMPRESS_OPCODE_CASES = [1, 2, 4, 8, 16, 32, 64, 128];
-
-class SPCOMPRESS_INST<int NumResultRegs, int NumSrc, int NumTgt>
+class SPCOMPRESS_INST<SPLayout Layout>
: NVPTXInst<(outs), (ins), "?", []> {
- let OutOperandList = SPRegList<NumResultRegs, "r">.Dag;
- let InOperandList = (ins SPQualifier:$elem_size,
- SPQualifier:$idx_size,
- SPQualifier:$repeat_factor,
- variable_ops);
+ SPRegList Results = SPRegList<Layout.NumResultRegs, "r">;
+ SPRegList Inputs = SPRegList<Layout.NumInputRegs, "i">;
+ SDNode Opcode = SPNode<"NVPTXISD::SPCOMPRESS_R" # Layout.NumResultRegs,
+ Layout, 5>;
+ list<string> OperandNames =
+ !listconcat(["elem_size", "idx_size", "repeat_factor", "num_src",
+ "num_tgt"],
+ Inputs.Names);
+ int NumOperands = !size(OperandNames);
+ dag FragmentQualifierPattern =
+ (Opcode node:$elem_size, node:$idx_size, node:$repeat_factor,
+ node:$num_src, node:$num_tgt);
+ dag FragmentInputPattern =
+ !dag(Opcode, !listsplat(node, Layout.NumInputRegs), Inputs.Names);
+ PatFrag Match = SPExactMatch<
+ !dag(ops, !listsplat(node, NumOperands), OperandNames),
+ !con(FragmentQualifierPattern, FragmentInputPattern), NumOperands>;
+ dag MatchQualifierPattern =
+ (Match timm:$elem_size, timm:$idx_size, timm:$repeat_factor,
+ timm:$num_src, timm:$num_tgt);
+ dag MatchInputPattern = !dag(Match, !listsplat(i32, Layout.NumInputRegs),
+ Inputs.Names);
+
+ let OutOperandList = Results.Outs;
+ let InOperandList = !con((ins SPModifiers:$elem_size,
+ SPModifiers:$idx_size,
+ SPModifiers:$repeat_factor,
+ SPModifiers:$num_src,
+ SPModifiers:$num_tgt),
+ Inputs.Ins);
let AsmString = "spcompress${elem_size:elem_size}${idx_size:idx_size}"
- # SPFactorName<NumSrc, NumTgt>.ret
- # "${repeat_factor:repeat_factor} "
+ "${num_src:spfactor}${repeat_factor:repeat_factor} "
"${elem_size:spcompress_ops};";
+ let Pattern = [!con(Results.Set,
+ (set !con(MatchQualifierPattern, MatchInputPattern)))];
let Predicates = [hasSPCompress];
}
-// SP qualifier operands control the PTX spelling and operand partitioning.
-// The opcode only encodes the number of physical result registers.
-foreach NumResultRegs = SPCOMPRESS_OPCODE_CASES in
- def SPCOMPRESS_OUT_ # NumResultRegs : SPCOMPRESS_INST<NumResultRegs, 2, 4>;
+foreach Layout = SPCOMPRESS_LAYOUT_SET in
+ def SPCOMPRESS_OUT_ # Layout.NumResultRegs # _IN_ # Layout.NumInputRegs
+ : SPCOMPRESS_INST<Layout>;
-class SPDECOMPRESS_INST<int NumResultRegs>
+class SPDECOMPRESS_INST<SPLayout Layout>
: NVPTXInst<(outs), (ins), "?", []> {
- let OutOperandList = SPRegList<NumResultRegs, "r">.Dag;
- let InOperandList = (ins SPQualifier:$elem_size,
- SPQualifier:$idx_size,
- SPQualifier:$repeat_factor,
- SPQualifier:$num_src,
- SPQualifier:$num_tgt,
- variable_ops);
+ SPRegList Results = SPRegList<Layout.NumResultRegs, "r">;
+ SPRegList Inputs = SPRegList<Layout.NumInputRegs, "i">;
+ SDNode Opcode = SPNode<"NVPTXISD::SPDECOMPRESS_R" # Layout.NumResultRegs,
+ Layout, 5>;
+ list<string> OperandNames =
+ !listconcat(["elem_size", "idx_size", "repeat_factor", "num_src",
+ "num_tgt"],
+ Inputs.Names);
+ int NumOperands = !size(OperandNames);
+ dag FragmentQualifierPattern =
+ (Opcode node:$elem_size, node:$idx_size, node:$repeat_factor,
+ node:$num_src, node:$num_tgt);
+ dag FragmentInputPattern =
+ !dag(Opcode, !listsplat(node, Layout.NumInputRegs), Inputs.Names);
+ PatFrag Match = SPExactMatch<
+ !dag(ops, !listsplat(node, NumOperands), OperandNames),
+ !con(FragmentQualifierPattern, FragmentInputPattern), NumOperands>;
+ dag MatchQualifierPattern =
+ (Match timm:$elem_size, timm:$idx_size, timm:$repeat_factor,
+ timm:$num_src, timm:$num_tgt);
+ dag MatchInputPattern = !dag(Match, !listsplat(i32, Layout.NumInputRegs),
+ Inputs.Names);
+
+ let OutOperandList = Results.Outs;
+ let InOperandList = !con((ins SPModifiers:$elem_size,
+ SPModifiers:$idx_size,
+ SPModifiers:$repeat_factor,
+ SPModifiers:$num_src,
+ SPModifiers:$num_tgt),
+ Inputs.Ins);
let AsmString = "spdecompress${elem_size:elem_size}${idx_size:idx_size}"
"${num_src:spfactor}${repeat_factor:repeat_factor} "
"${elem_size:spdecompress_ops};";
+ let Pattern = [!con(Results.Set,
+ (set !con(MatchQualifierPattern, MatchInputPattern)))];
let Predicates = [hasSPCompress];
}
-// SP qualifier operands control the PTX spelling and operand partitioning.
-// The opcode only encodes the number of physical result registers.
-foreach NumResultRegs = SPDECOMPRESS_OPCODE_CASES in
- def SPDECOMPRESS_OUT_ # NumResultRegs : SPDECOMPRESS_INST<NumResultRegs>;
+foreach Layout = SPDECOMPRESS_LAYOUT_SET in
+ def SPDECOMPRESS_OUT_ # Layout.NumResultRegs # _IN_ # Layout.NumInputRegs
+ : SPDECOMPRESS_INST<Layout>;
//
// tensormap.replace Instructions
diff --git a/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.cpp b/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.cpp
index c828747ce95451..710d063e75725c 100644
--- a/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.cpp
@@ -39,8 +39,6 @@ const char *NVPTXSelectionDAGInfo::getTargetNodeName(unsigned Opcode) const {
MAKE_CASE(NVPTXISD::SETP_F16X2)
MAKE_CASE(NVPTXISD::SETP_BF16X2)
MAKE_CASE(NVPTXISD::UNPACK_VECTOR)
- MAKE_CASE(NVPTXISD::SPCOMPRESS)
- MAKE_CASE(NVPTXISD::SPDECOMPRESS)
}
#undef MAKE_CASE
@@ -64,10 +62,6 @@ void NVPTXSelectionDAGInfo::verifyTargetNode(const SelectionDAG &DAG,
case NVPTXISD::ProxyReg:
// invalid number of results; expected 2, got 1
return;
- case NVPTXISD::SPCOMPRESS:
- case NVPTXISD::SPDECOMPRESS:
- // The packed register lists have variable numbers of operands and results.
- return;
}
return SelectionDAGGenTargetInfo::verifyTargetNode(DAG, N);
diff --git a/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h b/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h
index 8e29d08817e835..9dd0a1eaa5856b 100644
--- a/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h
+++ b/llvm/lib/Target/NVPTX/NVPTXSelectionDAGInfo.h
@@ -21,8 +21,6 @@ enum NodeType : unsigned {
SETP_F16X2 = GENERATED_OPCODE_END,
SETP_BF16X2,
UNPACK_VECTOR,
- SPCOMPRESS,
- SPDECOMPRESS,
FIRST_MEMORY_OPCODE,
>From 0aa974c4a06c6b9e051fd6bebdc29f74dfab1d57 Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Mon, 21 Sep 2026 06:58:57 +0200
Subject: [PATCH 10/10] [NVPTX] Fix code formatting
---
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
index dc10190c287805..4ff362b29d1754 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
@@ -3303,8 +3303,8 @@ static void appendSplitSPOperands(ArrayRef<SDUse> Input,
static unsigned getSPCompressNodeOpcode(unsigned NumResults) {
// SelectionDAG node descriptions have a fixed result arity, so use one node
// kind for each result-register count selected by the legal SP layouts.
-#define SPCOMPRESS_NODE_CASE(NumResults) \
- case NumResults: \
+#define SPCOMPRESS_NODE_CASE(NumResults) \
+ case NumResults: \
return NVPTXISD::SPCOMPRESS_R##NumResults
switch (NumResults) {
@@ -3332,8 +3332,8 @@ static unsigned getSPCompressNodeOpcode(unsigned NumResults) {
}
static unsigned getSPDecompressNodeOpcode(unsigned NumResults) {
-#define SPDECOMPRESS_NODE_CASE(NumResults) \
- case NumResults: \
+#define SPDECOMPRESS_NODE_CASE(NumResults) \
+ case NumResults: \
return NVPTXISD::SPDECOMPRESS_R##NumResults
switch (NumResults) {
@@ -3401,8 +3401,7 @@ static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
SDLoc DL(N);
EVT DataVT = N->getOperand(1).getValueType();
EVT CDataVT = N->getValueType(1);
- assert(DataVT.isVector() && CDataVT.isVector() &&
- "invalid spcompress types");
+ assert(DataVT.isVector() && CDataVT.isVector() && "invalid spcompress types");
unsigned ElemSize = DataVT.getScalarSizeInBits();
unsigned IdxSize = cast<ConstantSDNode>(N->getOperand(3))->getZExtValue();
More information about the llvm-commits
mailing list