[llvm] [NVVM][NVPTX] Support spcompress and spdecompress intrinsics (PR #221751)
Kirill Vedernikov via llvm-commits
llvm-commits at lists.llvm.org
Mon Sep 7 07:47:34 PDT 2026
https://github.com/kvederni created https://github.com/llvm/llvm-project/pull/221751
This change supports `spcompress` and `spdecompress` intrinsics and their lowering to the NVPTX backend.
>From 634512ec74ff2506a37bf0102c5b4a9d35bda12f Mon Sep 17 00:00:00 2001
From: Kirill Vedernikov <kvedernikov at nvidia.com>
Date: Mon, 7 Sep 2026 16:43:04 +0200
Subject: [PATCH] [NVVM][NVPTX] Support spcompress and spdecompress intrinsics
---
llvm/docs/NVPTXUsage.md | 108 +++
llvm/include/llvm/IR/IntrinsicsNVVM.td | 59 ++
llvm/include/llvm/IR/NVVMIntrinsicUtils.h | 132 +++
llvm/lib/IR/NVVMIntrinsicUtils.cpp | 17 +-
.../NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp | 111 +++
.../NVPTX/MCTargetDesc/NVPTXInstPrinter.h | 2 +
llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp | 199 ++++
llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp | 122 +++
llvm/lib/Target/NVPTX/NVPTXInstrInfo.td | 3 +
llvm/lib/Target/NVPTX/NVPTXIntrinsics.td | 70 ++
llvm/test/CodeGen/NVPTX/spcompress-invalid.ll | 83 ++
llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll | 860 ++++++++++++++++++
.../CodeGen/NVPTX/spdecompress-invalid.ll | 105 +++
.../CodeGen/NVPTX/spdecompress-sp1to16.ll | 309 +++++++
.../test/CodeGen/NVPTX/spdecompress-sp1to2.ll | 552 +++++++++++
.../test/CodeGen/NVPTX/spdecompress-sp1to4.ll | 682 ++++++++++++++
.../test/CodeGen/NVPTX/spdecompress-sp1to8.ll | 350 +++++++
.../CodeGen/NVPTX/spdecompress-sp2to16.ll | 325 +++++++
.../test/CodeGen/NVPTX/spdecompress-sp2to4.ll | 758 +++++++++++++++
.../test/CodeGen/NVPTX/spdecompress-sp2to8.ll | 379 ++++++++
.../CodeGen/NVPTX/spdecompress-sp4to16.ll | 184 ++++
.../test/CodeGen/NVPTX/spdecompress-sp4to8.ll | 224 +++++
22 files changed, 5633 insertions(+), 1 deletion(-)
create mode 100644 llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
create mode 100644 llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
diff --git a/llvm/docs/NVPTXUsage.md b/llvm/docs/NVPTXUsage.md
index d2ee19c42a602..58285e321747a 100644
--- a/llvm/docs/NVPTXUsage.md
+++ b/llvm/docs/NVPTXUsage.md
@@ -996,6 +996,114 @@ If the given pointer in the generic address space refers to memory which falls
within the state space of the intrinsic (and therefore could be safely address
space casted to this space), 1 is returned, otherwise 0 is returned.
+### Structured Sparsity Intrinsics
+
+The `llvm.nvvm.spcompress.*` and `llvm.nvvm.spdecompress.*` intrinsics model
+the PTX `spcompress` and `spdecompress` instructions. They require PTX ISA 9.4
+and `sm_107a`.
+
+The intrinsic types representing `mdata`, `cdata`, and `data` are overloaded.
+A bundle of one 32-bit register is represented by `i32`, and a bundle of `N`
+registers is represented by `<N x i32>`. The actual intrinsic names include
+the corresponding LLVM overload suffixes.
+
+The three trailing arguments are immediate qualifiers with the following
+encodings:
+
+| Argument | Values | PTX qualifiers |
+| ----------------------- | ------------------------- | -------------------------------------- |
+| `%elem_size` | `8`, `16` | `.b8`, `.b16` |
+| `%idx_size` | `2`, `4` | `.b2`, `.b4` |
+| `%repeat_factor` | `0`, `1`, ..., `6` | `.x1`, `.x2`, ..., `.x64` |
+
+The repeat factor `num` is `1 << %repeat_factor`.
+
+#### '`llvm.nvvm.spcompress.sp2to4`' Intrinsic
+
+##### Syntax:
+
+```llvm
+declare {MDataTy, CDataTy} @llvm.nvvm.spcompress.sp2to4(
+ DataTy %data, i32 %spdesc, i32 immarg %elem_size,
+ i32 immarg %idx_size, i32 immarg %repeat_factor)
+```
+
+##### Overview:
+
+This intrinsic compresses the dense vector `%data` using 2:4 structured
+sparsity. It returns the selected element indices as `mdata` and the selected
+elements as `cdata`. The number of 32-bit registers in each bundle is:
+
+| Bundle | Register count |
+| ------- | ------------------------------------------- |
+| `data` | `2 * num` |
+| `cdata` | `num` |
+| `mdata` | `ceil(num * %idx_size / %elem_size)` |
+
+The combined `mdata`, `cdata`, and `data` bundle size must not exceed 253
+registers.
+
+The `%spdesc` operand specifies the selection operation and the element data
+type:
+
+| Bits | Description | Values |
+| ---- | ------------------- | ------------------------------------------------------------------------- |
+| 0-1 | Selection operation | `0`: max, `1`: maxabs, `2`: min, `3`: minabs |
+| 2-4 | Element data type | `0`: f16/u8, `1`: bf16/s8, `2`: e5m2, `3`: e4m3, `4`: e3m2, `5`: e2m3 |
+| 5-31 | Reserved | `0` |
+
+The element data type must be consistent with `%elem_size`. When the metadata
+occupies less than 32 bits, it is zero-extended to fill its `i32` register.
+The operation treats negative zero as less than positive zero. NaN elements
+are always selected; when multiple selections satisfy the same comparison,
+the selected indices are implementation-specific.
+
+For more information, see the
+[PTX ISA](https://docs.nvidia.com/cuda/parallel-thread-execution/#data-movement-and-conversion-instructions-spcompress).
+
+#### '`llvm.nvvm.spdecompress.*`' Intrinsics
+
+##### Syntax:
+
+```llvm
+declare DataTy @llvm.nvvm.spdecompress.{sp1to2,sp1to4,sp1to8,sp1to16,
+ sp2to4,sp2to8,sp2to16,
+ sp4to8,sp4to16}(
+ MDataTy %mdata, CDataTy %cdata, i32 immarg %elem_size,
+ i32 immarg %idx_size, i32 immarg %repeat_factor)
+```
+
+##### Overview:
+
+These intrinsics decompress the structured sparse vector `%cdata` into a
+dense `data` vector. The intrinsic name specifies the number of source and
+target elements in each group. For example, `sp2to4` maps two compressed
+elements into a group of four dense elements using the indices in `%mdata`.
+Dense positions not selected by the metadata are set to zero.
+
+For an intrinsic named `spXtoY`, the number of 32-bit registers in each bundle
+is:
+
+| Bundle | Register count |
+| ------- | --------------------------------------------------- |
+| `mdata` | `ceil(X * %idx_size * num / 32)` |
+| `cdata` | `ceil(X * %elem_size * num / 32)` |
+| `data` | `ceil(Y * %elem_size * num / 32)` |
+
+The following conditions must hold:
+
+- `X * %elem_size <= 32`.
+- `%idx_size` is `2` only when `Y <= 4`.
+- `32 <= Y * %elem_size * num <= 4096`.
+- The combined `mdata`, `cdata`, and `data` bundle size does not exceed 253
+ registers.
+
+The behavior is implementation-specific if a metadata index does not identify
+a position in its target group.
+
+For more information, see the
+[PTX ISA](https://docs.nvidia.com/cuda/parallel-thread-execution/#data-movement-and-conversion-instructions-spdecompress).
+
### Narrow Floating-Point Conversion intrinsics
These intrinsics perform conversions involving narrow floating-point formats.
diff --git a/llvm/include/llvm/IR/IntrinsicsNVVM.td b/llvm/include/llvm/IR/IntrinsicsNVVM.td
index c50609f9fc5ee..a61615d1ffa3e 100644
--- a/llvm/include/llvm/IR/IntrinsicsNVVM.td
+++ b/llvm/include/llvm/IR/IntrinsicsNVVM.td
@@ -145,6 +145,10 @@ def llvm_local_ptr_ty : LLVMQualPointerType<5>; // (local)ptr
def llvm_tmem_ptr_ty : LLVMQualPointerType<6>; // (tensor memory)ptr
def llvm_shared_cluster_ptr_ty : LLVMQualPointerType<7>; // (shared_cluster)ptr
+// A RangeList is a RangeSet with singleton closed ranges for each listed value.
+class RangeList<ArgIndex idx, list<int> values>
+ : RangeSet<idx, !foreach(value, values, [value, value])>;
+
//
// MISC
//
@@ -1300,6 +1304,23 @@ class NVVMPureIntrinsic<list<LLVMType> ret_types,
DefaultAttrsIntrinsic<ret_types, param_types,
intr_properties # [IntrNoMem, IntrSpeculatable], name> {}
+defvar NVVM_SP_LG2_REPEAT_FACTOR_LIMIT = 7;
+
+class NVVM_SP_ARGPROPS<ArgIndex ElemSizeIdx, ArgIndex IdxSizeIdx,
+ ArgIndex RepeatFactorIdx> {
+ list<IntrinsicProperty> Prop = [
+ RangeList<ElemSizeIdx, [8, 16]>,
+ RangeList<IdxSizeIdx, [2, 4]>,
+ Range<RepeatFactorIdx, 0, NVVM_SP_LG2_REPEAT_FACTOR_LIMIT>,
+ ArgInfo<ElemSizeIdx, [ArgName<"elem_size">,
+ ImmArgPrinter<"printSPElemSize">]>,
+ ArgInfo<IdxSizeIdx, [ArgName<"idx_size">,
+ ImmArgPrinter<"printSPIdxSize">]>,
+ ArgInfo<RepeatFactorIdx, [ArgName<"repeat_factor">,
+ ImmArgPrinter<"printSPRepeatFactor">]>
+ ];
+}
+
// For im2col mode the number of offsets is (dim - 2).
// For im2col_w / im2col_w_128 modes it is always 2 (wHalo and wOffset),
// irrespective of dim.
@@ -3983,4 +4004,42 @@ def int_nvvm_tensormap_replace_fill_mode :
ArgInfo<ArgIndex<1>, [ArgName<"fill_mode">,
ImmArgPrinter<"printTensormapFillMode">]>]>;
+//
+// spcompress intrinsics
+//
+
+class SPCompress :
+ DefaultAttrsIntrinsicFlags<
+ [llvm_anyint_ty, // mdata
+ llvm_anyint_ty], // cdata
+ [llvm_anyint_ty, // data
+ llvm_i32_ty], // spdesc
+ [llvm_i32_ty, // elem_size
+ llvm_i32_ty, // idx_size
+ llvm_i32_ty], // repeat_factor: 0..6 for .x1 .. .x64
+ !listconcat([IntrNoMem, IntrSpeculatable],
+ NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>, ArgIndex<4>>.Prop)>;
+
+def int_nvvm_spcompress_sp2to4 : SPCompress;
+
+//
+// spdecompress intrinsics
+//
+
+class SPDecompress :
+ DefaultAttrsIntrinsicFlags<
+ [llvm_anyint_ty], // data
+ [llvm_anyint_ty, // mdata
+ llvm_anyint_ty], // cdata
+ [llvm_i32_ty, // elem_size
+ llvm_i32_ty, // idx_size
+ llvm_i32_ty], // repeat_factor: 0..6 for .x1 .. .x64
+ !listconcat([IntrNoMem, IntrSpeculatable],
+ NVVM_SP_ARGPROPS<ArgIndex<2>, ArgIndex<3>, ArgIndex<4>>.Prop)>;
+
+foreach num_src = [1, 2, 4] in
+ foreach num_tgt = [2, 4, 8, 16] in
+ if !lt(num_src, num_tgt) then
+ def int_nvvm_spdecompress_sp # num_src # to # num_tgt : SPDecompress;
+
} // let TargetPrefix = "nvvm"
diff --git a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
index 950f852ba820b..028abbd0f44a4 100644
--- a/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
+++ b/llvm/include/llvm/IR/NVVMIntrinsicUtils.h
@@ -19,6 +19,7 @@
#include "llvm/ADT/APFloat.h"
#include "llvm/ADT/APInt.h"
+#include "llvm/ADT/Enum.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/IR/Constants.h"
#include "llvm/IR/Intrinsics.h"
@@ -156,6 +157,133 @@ enum class TensormapFillMode : uint8_t {
OOB_NAN_FILL = 1,
};
+enum class SPElemSize : uint64_t {
+ B8 = 8,
+ B16 = 16,
+};
+
+enum class SPIdxSize : uint64_t {
+ B2 = 2,
+ B4 = 4,
+};
+
+enum class SPLg2RepeatFactor : uint64_t {
+ X1 = 0,
+ X2 = 1,
+ X4 = 2,
+ X8 = 3,
+ X16 = 4,
+ X32 = 5,
+ X64 = 6,
+};
+
+struct SPOperandLayout {
+ unsigned MetadataSize;
+ unsigned CompressedDataSize;
+ unsigned DataSize;
+};
+
+// PTX limits the combined vector size of the mdata, cdata, and data operands
+// of spcompress and spdecompress to 253 32-bit registers.
+inline constexpr unsigned MaxSPOperandRegisters = 253;
+
+inline StringRef getSPElemSizeName(SPElemSize ElemSize) {
+ static constexpr EnumStringDef<SPElemSize> SPElemSizeNameDefs[] = {
+ {{".b8"}, SPElemSize::B8},
+ {{".b16"}, SPElemSize::B16},
+ };
+ static constexpr auto SPElemSizeNames =
+ BUILD_ENUM_STRINGS(SPElemSizeNameDefs);
+ return EnumStrings(SPElemSizeNames).toString(ElemSize);
+}
+
+inline StringRef getSPIdxSizeName(SPIdxSize IdxSize) {
+ static constexpr EnumStringDef<SPIdxSize> SPIdxSizeNameDefs[] = {
+ {{".b2"}, SPIdxSize::B2},
+ {{".b4"}, SPIdxSize::B4},
+ };
+ static constexpr auto SPIdxSizeNames = BUILD_ENUM_STRINGS(SPIdxSizeNameDefs);
+ return EnumStrings(SPIdxSizeNames).toString(IdxSize);
+}
+
+inline StringRef getSPRepeatFactorName(SPLg2RepeatFactor Lg2RepeatFactor) {
+ static constexpr EnumStringDef<SPLg2RepeatFactor> SPRepeatFactorNameDefs[] = {
+ {{".x1"}, SPLg2RepeatFactor::X1}, {{".x2"}, SPLg2RepeatFactor::X2},
+ {{".x4"}, SPLg2RepeatFactor::X4}, {{".x8"}, SPLg2RepeatFactor::X8},
+ {{".x16"}, SPLg2RepeatFactor::X16}, {{".x32"}, SPLg2RepeatFactor::X32},
+ {{".x64"}, SPLg2RepeatFactor::X64},
+ };
+ static constexpr auto SPRepeatFactorNames =
+ BUILD_ENUM_STRINGS(SPRepeatFactorNameDefs);
+ return EnumStrings(SPRepeatFactorNames).toString(Lg2RepeatFactor);
+}
+
+inline unsigned getSPRepeatFactor(unsigned Lg2RepeatFactor) {
+ return 1U << Lg2RepeatFactor;
+}
+
+inline bool isValidSPDecompressFactor(unsigned NumSrc, unsigned NumTgt) {
+ switch (NumSrc) {
+ case 1:
+ return NumTgt == 2 || NumTgt == 4 || NumTgt == 8 || NumTgt == 16;
+ case 2:
+ return NumTgt == 4 || NumTgt == 8 || NumTgt == 16;
+ case 4:
+ return NumTgt == 8 || NumTgt == 16;
+ default:
+ return false;
+ }
+}
+
+inline std::optional<SPOperandLayout>
+getSPCompressLayout(unsigned ElemSize, unsigned IdxSize,
+ unsigned Lg2RepeatFactor) {
+ if (getSPElemSizeName(static_cast<SPElemSize>(ElemSize)).empty() ||
+ getSPIdxSizeName(static_cast<SPIdxSize>(IdxSize)).empty() ||
+ getSPRepeatFactorName(static_cast<SPLg2RepeatFactor>(Lg2RepeatFactor))
+ .empty())
+ return std::nullopt;
+
+ unsigned RepeatFactor = getSPRepeatFactor(Lg2RepeatFactor);
+ SPOperandLayout Layout = {
+ divideCeil(RepeatFactor * IdxSize, ElemSize),
+ RepeatFactor,
+ RepeatFactor * 2,
+ };
+ if (Layout.MetadataSize + Layout.CompressedDataSize + Layout.DataSize >
+ MaxSPOperandRegisters)
+ return std::nullopt;
+ return Layout;
+}
+
+inline std::optional<SPOperandLayout>
+getSPDecompressLayout(unsigned NumSrc, unsigned NumTgt, unsigned ElemSize,
+ unsigned IdxSize, unsigned Lg2RepeatFactor) {
+ if (!isValidSPDecompressFactor(NumSrc, NumTgt) ||
+ getSPElemSizeName(static_cast<SPElemSize>(ElemSize)).empty() ||
+ getSPIdxSizeName(static_cast<SPIdxSize>(IdxSize)).empty() ||
+ getSPRepeatFactorName(static_cast<SPLg2RepeatFactor>(Lg2RepeatFactor))
+ .empty())
+ return std::nullopt;
+ if (NumSrc * ElemSize > 32 || (IdxSize == 2 && NumTgt > 4))
+ return std::nullopt;
+
+ unsigned RepeatFactor = getSPRepeatFactor(Lg2RepeatFactor);
+ unsigned DataBits = NumTgt * ElemSize * RepeatFactor;
+ if (DataBits < 32 || DataBits > 4096)
+ return std::nullopt;
+
+ SPOperandLayout Layout = {
+ divideCeil(NumSrc * IdxSize * RepeatFactor, 32),
+ divideCeil(NumSrc * ElemSize * RepeatFactor, 32),
+ divideCeil(DataBits, 32),
+ };
+ if (Layout.MetadataSize + Layout.CompressedDataSize + Layout.DataSize >
+ MaxSPOperandRegisters)
+ return std::nullopt;
+ return Layout;
+}
+
LLVM_ABI void printTcgen05MMAKind(raw_ostream &OS, const Constant *ImmArgVal);
LLVM_ABI void printEvictPolicyType(raw_ostream &OS, const Constant *ImmArgVal);
@@ -168,6 +296,10 @@ LLVM_ABI void printTcgen05CollectorUsageOp(raw_ostream &OS,
LLVM_ABI void printTcgen05MMACollectorBBuffer(raw_ostream &OS,
const Constant *ImmArgVal);
+LLVM_ABI void printSPElemSize(raw_ostream &OS, const Constant *ImmArgVal);
+LLVM_ABI void printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal);
+LLVM_ABI void printSPRepeatFactor(raw_ostream &OS, const Constant *ImmArgVal);
+
LLVM_ABI void printTensormapElemType(raw_ostream &OS,
const Constant *ImmArgVal);
LLVM_ABI void printTensormapInterleaveLayout(raw_ostream &OS,
diff --git a/llvm/lib/IR/NVVMIntrinsicUtils.cpp b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
index 8069cf7eb6f1e..0d04b48dd6033 100644
--- a/llvm/lib/IR/NVVMIntrinsicUtils.cpp
+++ b/llvm/lib/IR/NVVMIntrinsicUtils.cpp
@@ -11,7 +11,6 @@
//===----------------------------------------------------------------------===//
#include "llvm/IR/NVVMIntrinsicUtils.h"
-#include "llvm/ADT/StringRef.h"
using namespace llvm;
using namespace nvvm;
@@ -101,6 +100,22 @@ void nvvm::printTcgen05MMACollectorBBuffer(raw_ostream &OS,
}
}
+void nvvm::printSPElemSize(raw_ostream &OS, const Constant *ImmArgVal) {
+ if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
+ OS << getSPElemSizeName(static_cast<SPElemSize>(CI->getZExtValue()));
+}
+
+void nvvm::printSPIdxSize(raw_ostream &OS, const Constant *ImmArgVal) {
+ if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
+ OS << getSPIdxSizeName(static_cast<SPIdxSize>(CI->getZExtValue()));
+}
+
+void nvvm::printSPRepeatFactor(raw_ostream &OS, const Constant *ImmArgVal) {
+ if (const auto *CI = dyn_cast<ConstantInt>(ImmArgVal))
+ OS << getSPRepeatFactorName(
+ static_cast<SPLg2RepeatFactor>(CI->getZExtValue()));
+}
+
void nvvm::printTensormapElemType(raw_ostream &OS, const Constant *ImmArgVal) {
static constexpr StringRef TensormapElemTypes[] = {
"u8", "u16", "u32", "s32", "u64", "s64",
diff --git a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
index 19855fd647eca..cfe4f4833f0a5 100644
--- a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
+++ b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.cpp
@@ -474,6 +474,117 @@ void NVPTXInstPrinter::printMmaCode(const MCInst *MI, int OpNum,
llvm_unreachable("Unknown Modifier");
}
+static void printSPRegList(NVPTXInstPrinter *Printer, const MCInst *MI,
+ unsigned FirstOp, unsigned NumOps,
+ const MCSubtargetInfo &STI, raw_ostream &O) {
+ O << "{";
+ for (unsigned I = 0; I != NumOps; ++I) {
+ if (I != 0)
+ O << ", ";
+ Printer->printOperand(MI, FirstOp + I, STI, O);
+ }
+ O << "}";
+}
+
+void NVPTXInstPrinter::printSPQualifier(const MCInst *MI, int OpNum,
+ const MCSubtargetInfo &STI,
+ raw_ostream &O, StringRef Modifier) {
+ if (Modifier.empty()) {
+ printOperand(MI, OpNum, STI, O);
+ return;
+ }
+
+ unsigned Imm = MI->getOperand(OpNum).getImm();
+ if (Modifier == "elem_size") {
+ StringRef ElemSize =
+ nvvm::getSPElemSizeName(static_cast<nvvm::SPElemSize>(Imm));
+ assert(!ElemSize.empty() && "invalid SP elem_size");
+ O << ElemSize;
+ return;
+ }
+
+ if (Modifier == "idx_size") {
+ StringRef IdxSize =
+ nvvm::getSPIdxSizeName(static_cast<nvvm::SPIdxSize>(Imm));
+ assert(!IdxSize.empty() && "invalid SP idx_size");
+ O << IdxSize;
+ return;
+ }
+
+ if (Modifier == "repeat_factor") {
+ StringRef RepeatFactor =
+ nvvm::getSPRepeatFactorName(static_cast<nvvm::SPLg2RepeatFactor>(Imm));
+ assert(!RepeatFactor.empty() && "invalid SP repeat_factor");
+ O << RepeatFactor;
+ return;
+ }
+
+ if (Modifier == "spfactor") {
+ unsigned NumTgt = MI->getOperand(OpNum + 1).getImm();
+ assert(nvvm::isValidSPDecompressFactor(Imm, NumTgt) && "invalid spfactor");
+ O << ".sp::" << Imm << ":" << NumTgt;
+ return;
+ }
+
+ auto GetSPLayout = [&](auto GetLayout) {
+ return GetLayout(Imm,
+ static_cast<unsigned>(MI->getOperand(OpNum + 1).getImm()),
+ static_cast<unsigned>(MI->getOperand(OpNum + 2).getImm()));
+ };
+
+ auto PrintSPRegLists = [&](unsigned FirstOp0, unsigned NumOps0,
+ unsigned FirstOp1, unsigned NumOps1,
+ unsigned FirstOp2, unsigned NumOps2) {
+ printSPRegList(this, MI, FirstOp0, NumOps0, STI, O);
+ O << ", ";
+ printSPRegList(this, MI, FirstOp1, NumOps1, STI, O);
+ O << ", ";
+ printSPRegList(this, MI, FirstOp2, NumOps2, STI, O);
+ };
+
+ if (Modifier == "spcompress_ops") {
+ auto Layout = GetSPLayout(nvvm::getSPCompressLayout);
+ assert(Layout && "invalid spcompress qualifiers");
+
+ unsigned FirstVariableOp = OpNum + 3;
+ assert(static_cast<unsigned>(OpNum) ==
+ Layout->MetadataSize + Layout->CompressedDataSize &&
+ MI->getNumOperands() == FirstVariableOp + Layout->DataSize + 1 &&
+ "invalid spcompress operand layout");
+
+ PrintSPRegLists(0, Layout->MetadataSize, Layout->MetadataSize,
+ Layout->CompressedDataSize, FirstVariableOp,
+ Layout->DataSize);
+ O << ", ";
+ printOperand(MI, FirstVariableOp + Layout->DataSize, STI, O);
+ return;
+ }
+
+ if (Modifier == "spdecompress_ops") {
+ unsigned NumSrc = MI->getOperand(OpNum + 3).getImm();
+ unsigned NumTgt = MI->getOperand(OpNum + 4).getImm();
+ auto Layout = GetSPLayout(
+ [&](unsigned ElemSize, unsigned IdxSize, unsigned Lg2RepeatFactor) {
+ return nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSize, IdxSize,
+ Lg2RepeatFactor);
+ });
+ assert(Layout && "invalid spdecompress qualifiers");
+
+ unsigned FirstVariableOp = OpNum + 5;
+ assert(static_cast<unsigned>(OpNum) == Layout->DataSize &&
+ MI->getNumOperands() == FirstVariableOp + Layout->MetadataSize +
+ Layout->CompressedDataSize &&
+ "invalid spdecompress operand layout");
+
+ PrintSPRegLists(0, Layout->DataSize, FirstVariableOp, Layout->MetadataSize,
+ FirstVariableOp + Layout->MetadataSize,
+ Layout->CompressedDataSize);
+ return;
+ }
+
+ llvm_unreachable(formatv("Unknown Modifier: {}", Modifier).str().c_str());
+}
+
void NVPTXInstPrinter::printMemOperand(const MCInst *MI, int OpNum,
const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier) {
diff --git a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
index d98b9998a9179..3105c3062d998 100644
--- a/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
+++ b/llvm/lib/Target/NVPTX/MCTargetDesc/NVPTXInstPrinter.h
@@ -53,6 +53,8 @@ class NVPTXInstPrinter : public MCInstPrinter {
raw_ostream &O);
void printMmaCode(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier = {});
+ void printSPQualifier(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
+ raw_ostream &O, StringRef Modifier = {});
void printMemOperand(const MCInst *MI, int OpNum, const MCSubtargetInfo &STI,
raw_ostream &O, StringRef Modifier = {});
void printUsedBytesMaskPragma(const MCInst *MI, int OpNum,
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
index 9a58a7108a85f..37dfc1b40d1a7 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelDAGToDAG.cpp
@@ -114,6 +114,7 @@ class NVPTXDAGToDAGISel : public SelectionDAGISel {
#include "NVPTXGenDAGISel.inc"
void Select(SDNode *N) override;
+ bool tryIntrinsicNoChain(SDNode *N);
bool tryIntrinsicChain(SDNode *N);
bool tryIntrinsicVoid(SDNode *N);
void SelectTexSurfHandle(SDNode *N);
@@ -137,6 +138,8 @@ class NVPTXDAGToDAGISel : public SelectionDAGISel {
bool IsIm2Col = false);
void SelectTcgen05Ld(SDNode *N, bool hasOffset = false);
void SelectTcgen05St(SDNode *N, bool hasOffset = false);
+ void selectSPCompress(SDNode *N);
+ void selectSPDecompress(SDNode *N, unsigned NumSrc, unsigned NumTgt);
void selectAtomicSwap128(SDNode *N);
inline SDValue getI32Imm(unsigned Imm, const SDLoc &DL) {
@@ -287,6 +290,10 @@ void NVPTXDAGToDAGISel::Select(SDNode *N) {
if (tryStoreVector(N))
return;
break;
+ case ISD::INTRINSIC_WO_CHAIN:
+ if (tryIntrinsicNoChain(N))
+ return;
+ break;
case ISD::INTRINSIC_W_CHAIN:
if (tryIntrinsicChain(N))
return;
@@ -439,6 +446,198 @@ void NVPTXDAGToDAGISel::SelectTcgen05Ld(SDNode *N, bool hasOffset) {
}
}
+static void checkSPRegTypes(SDNode *N, unsigned NumFlags, StringRef Name) {
+ for (const EVT VT : N->values())
+ if (VT != MVT::i32)
+ report_fatal_error(Twine(Name) + " expects scalarized i32 results");
+
+ for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
+ if (N->getOperand(I).getValueType() != MVT::i32)
+ report_fatal_error(Twine(Name) + " expects scalarized i32 operands");
+}
+
+// Keep these lists in sync with their counterparts in NVPTXIntrinsics.td. A
+// C++-only count references a nonexistent opcode and fails to compile; a
+// TableGen-only count reaches llvm_unreachable in the selectors below.
+#define SPCOMPRESS_OPCODE_CASES(CASE_MACRO) \
+ /* .b8.b2 and .b16.b4: ceil(repeat_factor / 4) + repeat_factor. */ \
+ /* The repeat_factor operand encodes the exponent. */ \
+ CASE_MACRO(2) /* .x1 */ \
+ CASE_MACRO(3) /* .x2 */ \
+ CASE_MACRO(5) /* .x4 */ \
+ CASE_MACRO(10) /* .x8 */ \
+ CASE_MACRO(20) /* .x16 */ \
+ CASE_MACRO(40) /* .x32 */ \
+ CASE_MACRO(80) /* .x64 */ \
+ /* .b16.b2: additional unique counts after the duplicate small cases. */ \
+ CASE_MACRO(9) /* .x8 */ \
+ CASE_MACRO(18) /* .x16 */ \
+ CASE_MACRO(36) /* .x32 */ \
+ CASE_MACRO(72) /* .x64 */ \
+ /* .b8.b4: additional unique counts after the duplicate small cases. */ \
+ CASE_MACRO(6) /* .x4 */ \
+ CASE_MACRO(12) /* .x8 */ \
+ CASE_MACRO(24) /* .x16 */ \
+ CASE_MACRO(48) /* .x32 */ \
+ CASE_MACRO(96) /* .x64 */
+
+#define SPDECOMPRESS_OPCODE_CASES(CASE_MACRO) \
+ CASE_MACRO(1) \
+ CASE_MACRO(2) \
+ CASE_MACRO(4) \
+ CASE_MACRO(8) \
+ CASE_MACRO(16) \
+ CASE_MACRO(32) \
+ CASE_MACRO(64) \
+ CASE_MACRO(128)
+
+void NVPTXDAGToDAGISel::selectSPCompress(SDNode *N) {
+ constexpr unsigned NumFlags = 3;
+ if (N->getNumOperands() < NumFlags + 2)
+ report_fatal_error("Malformed spcompress intrinsic");
+
+ unsigned FlagOp = N->getNumOperands() - NumFlags;
+ unsigned ElemSize =
+ cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
+ unsigned IdxSize =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
+ unsigned Lg2RepeatFactor =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 2))->getZExtValue();
+
+ auto Layout = nvvm::getSPCompressLayout(ElemSize, IdxSize, Lg2RepeatFactor);
+ if (!Layout)
+ report_fatal_error(Twine("Invalid spcompress flags: elem_size=") +
+ Twine(ElemSize) + ", idx_size=" + Twine(IdxSize) +
+ ", repeat_factor=" + Twine(Lg2RepeatFactor));
+ checkSPRegTypes(N, NumFlags, "spcompress");
+
+ if (N->getNumValues() != Layout->MetadataSize + Layout->CompressedDataSize ||
+ N->getNumOperands() != Layout->DataSize + NumFlags + 2)
+ report_fatal_error("spcompress operand/result types do not match "
+ "the SP intrinsic flags");
+
+ SDLoc DL(N);
+ SmallVector<SDValue, 16> Operands;
+ Operands.reserve(N->getNumOperands() - NumFlags + 2);
+ Operands.push_back(getI32Imm(ElemSize, DL));
+ Operands.push_back(getI32Imm(IdxSize, DL));
+ Operands.push_back(getI32Imm(Lg2RepeatFactor, DL));
+ for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
+ Operands.push_back(N->getOperand(I));
+
+#define REPLACE_SPCOMPRESS_NODE(NumResults) \
+ case NumResults: \
+ ReplaceNode(N, CurDAG->getMachineNode(NVPTX::SPCOMPRESS_OUT_##NumResults, \
+ DL, N->getVTList(), Operands)); \
+ return;
+
+ switch (Layout->MetadataSize + Layout->CompressedDataSize) {
+ SPCOMPRESS_OPCODE_CASES(REPLACE_SPCOMPRESS_NODE)
+ default:
+ llvm_unreachable("invalid spcompress result count");
+ }
+
+#undef REPLACE_SPCOMPRESS_NODE
+}
+
+void NVPTXDAGToDAGISel::selectSPDecompress(SDNode *N, unsigned NumSrc,
+ unsigned NumTgt) {
+ constexpr unsigned NumFlags = 3;
+ if (N->getNumOperands() < NumFlags + 3)
+ report_fatal_error("Malformed spdecompress intrinsic");
+
+ unsigned FlagOp = N->getNumOperands() - NumFlags;
+ unsigned ElemSize =
+ cast<ConstantSDNode>(N->getOperand(FlagOp))->getZExtValue();
+ unsigned IdxSize =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 1))->getZExtValue();
+ unsigned Lg2RepeatFactor =
+ cast<ConstantSDNode>(N->getOperand(FlagOp + 2))->getZExtValue();
+
+ auto Layout = nvvm::getSPDecompressLayout(NumSrc, NumTgt, ElemSize, IdxSize,
+ Lg2RepeatFactor);
+ if (!Layout)
+ report_fatal_error(Twine("Invalid spdecompress flags: num_src=") +
+ Twine(NumSrc) + ", num_tgt=" + Twine(NumTgt) +
+ ", elem_size=" + Twine(ElemSize) +
+ ", idx_size=" + Twine(IdxSize) +
+ ", repeat_factor=" + Twine(Lg2RepeatFactor));
+ checkSPRegTypes(N, NumFlags, "spdecompress");
+
+ if (N->getNumValues() != Layout->DataSize ||
+ N->getNumOperands() !=
+ Layout->MetadataSize + Layout->CompressedDataSize + NumFlags + 1)
+ report_fatal_error("spdecompress operand/result types do not match "
+ "the SP intrinsic flags");
+
+ SDLoc DL(N);
+ SmallVector<SDValue, 16> Operands;
+ Operands.reserve(N->getNumOperands() - NumFlags + 4);
+ Operands.push_back(getI32Imm(ElemSize, DL));
+ Operands.push_back(getI32Imm(IdxSize, DL));
+ Operands.push_back(getI32Imm(Lg2RepeatFactor, DL));
+ Operands.push_back(getI32Imm(NumSrc, DL));
+ Operands.push_back(getI32Imm(NumTgt, DL));
+ for (unsigned I = 1, E = N->getNumOperands() - NumFlags; I != E; ++I)
+ Operands.push_back(N->getOperand(I));
+
+#define REPLACE_SPDECOMPRESS_NODE(NumResults) \
+ case NumResults: \
+ ReplaceNode(N, \
+ CurDAG->getMachineNode(NVPTX::SPDECOMPRESS_OUT_##NumResults, \
+ DL, N->getVTList(), Operands)); \
+ return;
+
+ switch (Layout->DataSize) {
+ SPDECOMPRESS_OPCODE_CASES(REPLACE_SPDECOMPRESS_NODE)
+ default:
+ llvm_unreachable("invalid spdecompress result count");
+ }
+
+#undef REPLACE_SPDECOMPRESS_NODE
+}
+
+#undef SPDECOMPRESS_OPCODE_CASES
+#undef SPCOMPRESS_OPCODE_CASES
+
+bool NVPTXDAGToDAGISel::tryIntrinsicNoChain(SDNode *N) {
+ unsigned IID = N->getConstantOperandVal(0);
+ switch (IID) {
+ default:
+ return false;
+ case Intrinsic::nvvm_spcompress_sp2to4:
+ selectSPCompress(N);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp1to2:
+ selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/2);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp1to4:
+ selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/4);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp1to8:
+ selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/8);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp1to16:
+ selectSPDecompress(N, /*NumSrc=*/1, /*NumTgt=*/16);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp2to4:
+ selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/4);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp2to8:
+ selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/8);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp2to16:
+ selectSPDecompress(N, /*NumSrc=*/2, /*NumTgt=*/16);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp4to8:
+ selectSPDecompress(N, /*NumSrc=*/4, /*NumTgt=*/8);
+ return true;
+ case Intrinsic::nvvm_spdecompress_sp4to16:
+ selectSPDecompress(N, /*NumSrc=*/4, /*NumTgt=*/16);
+ return true;
+ }
+}
+
bool NVPTXDAGToDAGISel::tryIntrinsicChain(SDNode *N) {
unsigned IID = N->getConstantOperandVal(1);
switch (IID) {
diff --git a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
index a946ceaa35d81..649f7ebb12e46 100644
--- a/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
+++ b/llvm/lib/Target/NVPTX/NVPTXISelLowering.cpp
@@ -3197,6 +3197,97 @@ static SDValue lowerIntrinsicWChain(SDValue Op, SelectionDAG &DAG) {
}
}
+static bool needsSPCompressScalarization(SDNode *N) {
+ bool NeedsScalarize =
+ N->getValueType(0).isVector() || N->getValueType(1).isVector();
+ for (const SDValue &Operand : N->ops())
+ NeedsScalarize |= Operand.getValueType().isVector();
+ return NeedsScalarize;
+}
+
+static SDValue lowerSPCompress(SDValue Op, SelectionDAG &DAG) {
+ SDNode *N = Op.getNode();
+ SDLoc DL(N);
+
+ if (!needsSPCompressScalarization(N))
+ return Op;
+
+ SmallVector<SDValue, 4> Ops;
+ for (const SDValue &Operand : N->ops()) {
+ if (Operand.getValueType().isVector())
+ DAG.ExtractVectorElements(Operand, Ops);
+ else
+ Ops.push_back(Operand);
+ }
+
+ if (N->getValueType(0).isVector() || N->getValueType(1).isVector()) {
+ SmallVector<EVT, 4> ResultTys;
+ for (const EVT VT : N->values()) {
+ if (VT.isVector())
+ ResultTys.append(VT.getVectorNumElements(), VT.getVectorElementType());
+ else
+ ResultTys.push_back(VT);
+ }
+
+ SDValue Lowered = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResultTys, Ops);
+
+ SmallVector<SDValue, 2> Retvals;
+ for (unsigned NewI = 0, I = 0, E = N->getNumValues(); I != E; ++I) {
+ if (EVT VT = N->getValueType(I); VT.isVector()) {
+ SmallVector<SDValue> Elements;
+ for (auto NewE = NewI + VT.getVectorNumElements(); NewI != NewE; ++NewI)
+ Elements.push_back(Lowered.getValue(NewI));
+ Retvals.push_back(DAG.getBuildVector(VT, DL, Elements));
+ } else {
+ Retvals.push_back(Lowered.getValue(NewI));
+ ++NewI;
+ }
+ }
+
+ return DAG.getMergeValues(Retvals, DL);
+ }
+
+ return DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, N->getVTList(), Ops);
+}
+
+static SDValue lowerSPDecompress(SDValue Op, SelectionDAG &DAG) {
+ SDNode *N = Op.getNode();
+ SDLoc DL(N);
+ EVT ResVT0 = N->getValueType(0);
+ // Every valid layout with a scalar data result also has scalar mdata and
+ // cdata operands, so only vector results require scalarization here.
+ if (!ResVT0.isVector())
+ return Op;
+
+ unsigned NumElts = ResVT0.getVectorNumElements();
+ EVT EltVT = ResVT0.getVectorElementType();
+ SmallVector<EVT, 4> ListVTs(NumElts, EltVT);
+ SDVTList ResVTs = DAG.getVTList(ListVTs);
+
+ SmallVector<SDValue, 4> Ops;
+ Ops.push_back(N->getOperand(0));
+
+ for (unsigned I = 1; I < N->getNumOperands(); ++I) {
+ SDValue Op = N->getOperand(I);
+ EVT OpVT = Op.getValueType();
+ if (OpVT.isVector()) {
+ for (unsigned J = 0, E = OpVT.getVectorNumElements(); J != E; ++J)
+ Ops.push_back(DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL,
+ OpVT.getVectorElementType(), Op,
+ DAG.getIntPtrConstant(J, DL)));
+ } else {
+ Ops.push_back(Op);
+ }
+ }
+
+ SDValue NewNode = DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, ResVTs, Ops);
+
+ SmallVector<SDValue, 4> ScalarRes;
+ for (unsigned I = 0; I < NumElts; ++I)
+ ScalarRes.push_back(NewNode.getValue(I));
+ return DAG.getBuildVector(ResVT0, DL, ScalarRes);
+}
+
static SDValue lowerIntrinsicWOChain(SDValue Op, SelectionDAG &DAG) {
switch (Op->getConstantOperandVal(0)) {
default:
@@ -3225,6 +3316,20 @@ static SDValue lowerIntrinsicWOChain(SDValue Op, SelectionDAG &DAG) {
case Intrinsic::nvvm_f32x4_to_e2m1x4_rs_satfinite:
case Intrinsic::nvvm_f32x4_to_e2m1x4_rs_relu_satfinite:
return lowerCvtRSIntrinsics(Op, DAG);
+
+ case Intrinsic::nvvm_spcompress_sp2to4:
+ return lowerSPCompress(Op, DAG);
+
+ case Intrinsic::nvvm_spdecompress_sp1to2:
+ case Intrinsic::nvvm_spdecompress_sp1to4:
+ case Intrinsic::nvvm_spdecompress_sp1to8:
+ case Intrinsic::nvvm_spdecompress_sp1to16:
+ case Intrinsic::nvvm_spdecompress_sp2to4:
+ case Intrinsic::nvvm_spdecompress_sp2to8:
+ case Intrinsic::nvvm_spdecompress_sp2to16:
+ case Intrinsic::nvvm_spdecompress_sp4to8:
+ case Intrinsic::nvvm_spdecompress_sp4to16:
+ return lowerSPDecompress(Op, DAG);
}
}
@@ -7297,6 +7402,23 @@ static SDValue combineIntrinsicWOChain(SDNode *N,
return diagnoseUnsupportedFAdd(N, DCI.DAG, IID, RoundingMode);
return combineFAddWithNeg(N, DCI.DAG, IID, RoundingMode);
}
+ case Intrinsic::nvvm_spcompress_sp2to4:
+ if (needsSPCompressScalarization(N))
+ return lowerSPCompress(SDValue(N, 0), DCI.DAG);
+ break;
+ case Intrinsic::nvvm_spdecompress_sp1to2:
+ case Intrinsic::nvvm_spdecompress_sp1to4:
+ case Intrinsic::nvvm_spdecompress_sp1to8:
+ case Intrinsic::nvvm_spdecompress_sp1to16:
+ case Intrinsic::nvvm_spdecompress_sp2to4:
+ case Intrinsic::nvvm_spdecompress_sp2to8:
+ case Intrinsic::nvvm_spdecompress_sp2to16:
+ case Intrinsic::nvvm_spdecompress_sp4to8:
+ case Intrinsic::nvvm_spdecompress_sp4to16: {
+ if (N->getValueType(0).isVector())
+ return lowerSPDecompress(SDValue(N, 0), DCI.DAG);
+ break;
+ }
}
return SDValue();
}
diff --git a/llvm/lib/Target/NVPTX/NVPTXInstrInfo.td b/llvm/lib/Target/NVPTX/NVPTXInstrInfo.td
index 846bcc0557464..095cf5ba314e2 100644
--- a/llvm/lib/Target/NVPTX/NVPTXInstrInfo.td
+++ b/llvm/lib/Target/NVPTX/NVPTXInstrInfo.td
@@ -183,6 +183,9 @@ def hasTensormapReplaceSupport : SubtargetPredicate;
// - conversions involving ue5m3x2
def hasRubinFamilySupport : PredOr<[SM107f]>;
+// Checks sparse compression/decompression instruction support.
+def hasSPCompress : PredOr<[SM107a]>;
+
// Checks tcgen05.shift instruction support.
def hasTcgen05ShiftSupport : PredOr<[SM100a, SM103a, SM110a]>;
diff --git a/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td b/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
index f41a1fffc187d..e9eff3d85e9ef 100644
--- a/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
+++ b/llvm/lib/Target/NVPTX/NVPTXIntrinsics.td
@@ -7240,6 +7240,76 @@ foreach space = ["tensor", "shared"] in {
} // cta_group
} // space
+def SPQualifier : Operand<i32> {
+ let PrintMethod = "printSPQualifier";
+}
+
+class SPRegList<int NumRegs, string Prefix> {
+ dag Dag = !dag(outs, !listsplat(B32, NumRegs),
+ !foreach(I, !range(NumRegs), Prefix # I));
+}
+
+class SPFactorName<int NumSrc, int NumTgt> {
+ string ret = ".sp::" # NumSrc # ":" # NumTgt;
+}
+
+// mdata + cdata = ceil(repeat_factor * idx_size / elem_size) + repeat_factor,
+// where the repeat_factor operand encodes the exponent.
+// The first group is shared by .b8.b2 and .b16.b4. The next groups are the
+// additional unique counts for .b16.b2 and .b8.b4.
+defvar SPCOMPRESS_OPCODE_CASES =
+ !sort(NumResultRegs,
+ !listconcat([2, 3, 5, 10, 20, 40, 80], [9, 18, 36, 72],
+ [6, 12, 24, 48, 96]),
+ NumResultRegs);
+
+// data = ceil(num_tgt * elem_size * repeat_factor / 32), where repeat_factor is
+// encoded as an exponent. The legal spdecompress layouts produce all
+// power-of-two data register counts from 1 to 128.
+defvar SPDECOMPRESS_OPCODE_CASES =
+ !listconcat(!foreach(lg2_rpf, !range(0, NVVM_SP_LG2_REPEAT_FACTOR_LIMIT),
+ !shl(1, lg2_rpf)),
+ [128]);
+
+class SPCOMPRESS_INST<int NumResultRegs, int NumSrc, int NumTgt>
+ : NVPTXInst<(outs), (ins), "?", []> {
+ let OutOperandList = SPRegList<NumResultRegs, "r">.Dag;
+ let InOperandList = (ins SPQualifier:$elem_size,
+ SPQualifier:$idx_size,
+ SPQualifier:$repeat_factor,
+ variable_ops);
+ let AsmString = "spcompress${elem_size:elem_size}${idx_size:idx_size}"
+ # SPFactorName<NumSrc, NumTgt>.ret
+ # "${repeat_factor:repeat_factor} "
+ "${elem_size:spcompress_ops};";
+ let Predicates = [hasSPCompress];
+}
+
+// SP qualifier operands control the PTX spelling and operand partitioning.
+// The opcode only encodes the number of physical result registers.
+foreach NumResultRegs = SPCOMPRESS_OPCODE_CASES in
+ def SPCOMPRESS_OUT_ # NumResultRegs : SPCOMPRESS_INST<NumResultRegs, 2, 4>;
+
+class SPDECOMPRESS_INST<int NumResultRegs>
+ : NVPTXInst<(outs), (ins), "?", []> {
+ let OutOperandList = SPRegList<NumResultRegs, "r">.Dag;
+ let InOperandList = (ins SPQualifier:$elem_size,
+ SPQualifier:$idx_size,
+ SPQualifier:$repeat_factor,
+ SPQualifier:$num_src,
+ SPQualifier:$num_tgt,
+ variable_ops);
+ let AsmString = "spdecompress${elem_size:elem_size}${idx_size:idx_size}"
+ "${num_src:spfactor}${repeat_factor:repeat_factor} "
+ "${elem_size:spdecompress_ops};";
+ let Predicates = [hasSPCompress];
+}
+
+// SP qualifier operands control the PTX spelling and operand partitioning.
+// The opcode only encodes the number of physical result registers.
+foreach NumResultRegs = SPDECOMPRESS_OPCODE_CASES in
+ def SPDECOMPRESS_OUT_ # NumResultRegs : SPDECOMPRESS_INST<NumResultRegs>;
+
//
// tensormap.replace Instructions
//
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
new file mode 100644
index 0000000000000..4f5113f0196e1
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spcompress-invalid.ll
@@ -0,0 +1,83 @@
+; RUN: split-file %s %t
+; RUN: not llvm-as -disable-output < %t/bad-elem-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
+; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
+; RUN: not llvm-as -disable-output < %t/bad-repeat-factor.ll 2>&1 | FileCheck %s --check-prefix=BAD-RPF
+; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
+; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
+; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
+; RUN: not --crash llc < %t/bad-result-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
+; RUN: not --crash llc < %t/bad-data-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA-TY
+
+;--- bad-elem-size.ll
+
+define { i32, i32 } @bad_spcompress_elem_size(i32 %spdesc,
+ <2 x i32> %data) {
+; BAD-ELEM: immarg value 9 for arg 2 out of range set
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 9, i32 2, i32 0)
+ ret { i32, i32 } %res
+}
+
+;--- bad-idx-size.ll
+
+define { i32, i32 } @bad_spcompress_idx_size(i32 %spdesc,
+ <2 x i32> %data) {
+; BAD-IDX: immarg value 3 for arg 3 out of range set
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 3, i32 0)
+ ret { i32, i32 } %res
+}
+
+;--- bad-repeat-factor.ll
+
+define { i32, i32 } @bad_spcompress_repeat_factor(i32 %spdesc,
+ <2 x i32> %data) {
+; BAD-RPF: immarg value 7 for arg 4 out of range [0,7)
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 7)
+ ret { i32, i32 } %res
+}
+
+;--- bad-mdata-size.ll
+
+define i32 @bad_spcompress_metadata_register_count(i32 %spdesc,
+ <8 x i32> %data) {
+; BAD-MDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %mdata = extractvalue { i32, <4 x i32> } %res, 0
+ ret i32 %mdata
+}
+
+;--- bad-cdata-size.ll
+
+define <2 x i32> @bad_spcompress_compressed_data_register_count(
+ i32 %spdesc, <8 x i32> %data) {
+; BAD-CDATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { <2 x i32>, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v2i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %cdata = extractvalue { <2 x i32>, <2 x i32> } %res, 1
+ ret <2 x i32> %cdata
+}
+
+;--- bad-data-size.ll
+
+define <4 x i32> @bad_spcompress_data_register_count(i32 %spdesc,
+ <4 x i32> %data) {
+; BAD-DATA: LLVM ERROR: spcompress operand/result types do not match the SP intrinsic flags
+ %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ %cdata = extractvalue { <2 x i32>, <4 x i32> } %res, 1
+ ret <4 x i32> %cdata
+}
+
+;--- bad-result-type.ll
+
+define i64 @bad_spcompress_result_type(i32 %spdesc, <2 x i32> %data) {
+; BAD-RESULT-TY: LLVM ERROR: spcompress expects scalarized i32 results
+ %res = call { i64, i32 } @llvm.nvvm.spcompress.sp2to4.i64.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ %mdata = extractvalue { i64, i32 } %res, 0
+ ret i64 %mdata
+}
+
+;--- bad-data-type.ll
+
+define { i32, i32 } @bad_spcompress_data_type(i32 %spdesc, i64 %data) {
+; BAD-DATA-TY: LLVM ERROR: spcompress expects scalarized i32 operands
+ %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.i64(i64 %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ ret { i32, i32 } %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
new file mode 100644
index 0000000000000..cc92575a3bbea
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spcompress-sp2to4.ll
@@ -0,0 +1,860 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32> %data) {
+; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<6>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x1 {%r4}, {%r5}, {%r2, %r3}, %r1;
+; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
+; CHECK-NEXT: ret;
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 0)
+ ret {i32, i32} %res
+}
+
+define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(i32 %spdesc, <2 x i32> %data) {
+; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<6>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x1 {%r4}, {%r5}, {%r2, %r3}, %r1;
+; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
+; CHECK-NEXT: ret;
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 0)
+ ret {i32, i32} %res
+}
+
+define {i32, i32} @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32> %data) {
+; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<6>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x1 {%r4}, {%r5}, {%r2, %r3}, %r1;
+; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
+; CHECK-NEXT: ret;
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 0)
+ ret {i32, i32} %res
+}
+
+define {i32, i32} @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(i32 %spdesc, <2 x i32> %data) {
+; FORMAT-LABEL: define { i32, i32 } @test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call { i32, i32 } @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<6>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x1 {%r4}, {%r5}, {%r2, %r3}, %r1;
+; CHECK-NEXT: st.param::func.b32 [func_retval0+4], %r5;
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r4;
+; CHECK-NEXT: ret;
+ %res = call {i32, i32} @llvm.nvvm.spcompress.sp2to4.i32.i32.v2i32(<2 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 0)
+ ret {i32, i32} %res
+}
+
+define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(i32 %spdesc, <4 x i32> %data) {
+; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<9>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x2 {%r6}, {%r7, %r8}, {%r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
+; CHECK-NEXT: ret;
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 1)
+ ret {i32, <2 x i32>} %res
+}
+
+define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(i32 %spdesc, <4 x i32> %data) {
+; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<9>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x2 {%r6}, {%r7, %r8}, {%r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
+; CHECK-NEXT: ret;
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 1)
+ ret {i32, <2 x i32>} %res
+}
+
+define {i32, <2 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(i32 %spdesc, <4 x i32> %data) {
+; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<9>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x2 {%r6}, {%r7, %r8}, {%r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
+; CHECK-NEXT: ret;
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 1)
+ ret {i32, <2 x i32>} %res
+}
+
+define {i32, <2 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(i32 %spdesc, <4 x i32> %data) {
+; FORMAT-LABEL: define { i32, <2 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call { i32, <2 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<9>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x2 {%r6}, {%r7, %r8}, {%r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0+8], {%r7, %r8};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r6;
+; CHECK-NEXT: ret;
+ %res = call {i32, <2 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v2i32.v4i32(<4 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 1)
+ ret {i32, <2 x i32>} %res
+}
+
+define {i32, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(i32 %spdesc, <8 x i32> %data) {
+; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_4_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x4 {%r10}, {%r11, %r12, %r13, %r14}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
+; CHECK-NEXT: ret;
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 2)
+ ret {i32, <4 x i32>} %res
+}
+
+define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(i32 %spdesc, <8 x i32> %data) {
+; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_4_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x4 {%r10}, {%r11, %r12, %r13, %r14}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
+; CHECK-NEXT: ret;
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 2)
+ ret {i32, <4 x i32>} %res
+}
+
+define {<2 x i32>, <4 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(i32 %spdesc, <8 x i32> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <4 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call { <2 x i32>, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<16>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_4_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x4 {%r10, %r11}, {%r12, %r13, %r14, %r15}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r10, %r11};
+; CHECK-NEXT: ret;
+ %res = call {<2 x i32>, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 2)
+ ret {<2 x i32>, <4 x i32>} %res
+}
+
+define {i32, <4 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(i32 %spdesc, <8 x i32> %data) {
+; FORMAT-LABEL: define { i32, <4 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call { i32, <4 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_4_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x4 {%r10}, {%r11, %r12, %r13, %r14}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r10;
+; CHECK-NEXT: ret;
+ %res = call {i32, <4 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v4i32.v8i32(<8 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 2)
+ ret {i32, <4 x i32>} %res
+}
+
+define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(i32 %spdesc, <16 x i32> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <8 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<28>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x8 {%r18, %r19}, {%r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27}, {%r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r24, %r25, %r26, %r27};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
+; CHECK-NEXT: ret;
+ %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 3)
+ ret {<2 x i32>, <8 x i32>} %res
+}
+
+define {i32, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(i32 %spdesc, <16 x i32> %data) {
+; FORMAT-LABEL: define { i32, <8 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call { i32, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x8 {%r18}, {%r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r18;
+; CHECK-NEXT: ret;
+ %res = call {i32, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 3)
+ ret {i32, <8 x i32>} %res
+}
+
+define {<4 x i32>, <8 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(i32 %spdesc, <16 x i32> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <8 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call { <4 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<30>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x8 {%r18, %r19, %r20, %r21}, {%r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29}, {%r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: ret;
+ %res = call {<4 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 3)
+ ret {<4 x i32>, <8 x i32>} %res
+}
+
+define {<2 x i32>, <8 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(i32 %spdesc, <16 x i32> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <8 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call { <2 x i32>, <8 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<28>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x8 {%r18, %r19}, {%r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27}, {%r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r24, %r25, %r26, %r27};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r20, %r21, %r22, %r23};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r18, %r19};
+; CHECK-NEXT: ret;
+ %res = call {<2 x i32>, <8 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v8i32.v16i32(<16 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 3)
+ ret {<2 x i32>, <8 x i32>} %res
+}
+
+define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(i32 %spdesc, <32 x i32> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <16 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<54>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x16 {%r34, %r35, %r36, %r37}, {%r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53}, {%r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r50, %r51, %r52, %r53};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r46, %r47, %r48, %r49};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r42, %r43, %r44, %r45};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: ret;
+ %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 4)
+ ret {<4 x i32>, <16 x i32>} %res
+}
+
+define {<2 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(i32 %spdesc, <32 x i32> %data) {
+; FORMAT-LABEL: define { <2 x i32>, <16 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call { <2 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<52>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x16 {%r34, %r35}, {%r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51}, {%r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r48, %r49, %r50, %r51};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r44, %r45, %r46, %r47};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r40, %r41, %r42, %r43};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r36, %r37, %r38, %r39};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r34, %r35};
+; CHECK-NEXT: ret;
+ %res = call {<2 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v2i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 4)
+ ret {<2 x i32>, <16 x i32>} %res
+}
+
+define {<8 x i32>, <16 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(i32 %spdesc, <32 x i32> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <16 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call { <8 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<58>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x16 {%r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41}, {%r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57}, {%r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r54, %r55, %r56, %r57};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r50, %r51, %r52, %r53};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r46, %r47, %r48, %r49};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r42, %r43, %r44, %r45};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: ret;
+ %res = call {<8 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 4)
+ ret {<8 x i32>, <16 x i32>} %res
+}
+
+define {<4 x i32>, <16 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(i32 %spdesc, <32 x i32> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <16 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call { <4 x i32>, <16 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<54>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x16 {%r34, %r35, %r36, %r37}, {%r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53}, {%r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r50, %r51, %r52, %r53};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r46, %r47, %r48, %r49};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r42, %r43, %r44, %r45};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: ret;
+ %res = call {<4 x i32>, <16 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v16i32.v32i32(<32 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 4)
+ ret {<4 x i32>, <16 x i32>} %res
+}
+
+define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(i32 %spdesc, <64 x i32> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <32 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<106>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x32 {%r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73}, {%r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105}, {%r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r102, %r103, %r104, %r105};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r98, %r99, %r100, %r101};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r94, %r95, %r96, %r97};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r90, %r91, %r92, %r93};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r86, %r87, %r88, %r89};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r82, %r83, %r84, %r85};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r78, %r79, %r80, %r81};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r74, %r75, %r76, %r77};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: ret;
+ %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 5)
+ ret {<8 x i32>, <32 x i32>} %res
+}
+
+define {<4 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(i32 %spdesc, <64 x i32> %data) {
+; FORMAT-LABEL: define { <4 x i32>, <32 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call { <4 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<102>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x32 {%r66, %r67, %r68, %r69}, {%r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101}, {%r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r98, %r99, %r100, %r101};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r94, %r95, %r96, %r97};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r90, %r91, %r92, %r93};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r86, %r87, %r88, %r89};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r82, %r83, %r84, %r85};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r78, %r79, %r80, %r81};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r74, %r75, %r76, %r77};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r70, %r71, %r72, %r73};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: ret;
+ %res = call {<4 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v4i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 5)
+ ret {<4 x i32>, <32 x i32>} %res
+}
+
+define {<16 x i32>, <32 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(i32 %spdesc, <64 x i32> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <32 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call { <16 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<114>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x32 {%r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81}, {%r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113}, {%r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r110, %r111, %r112, %r113};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r106, %r107, %r108, %r109};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r102, %r103, %r104, %r105};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r98, %r99, %r100, %r101};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r94, %r95, %r96, %r97};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r90, %r91, %r92, %r93};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r86, %r87, %r88, %r89};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r82, %r83, %r84, %r85};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r78, %r79, %r80, %r81};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r74, %r75, %r76, %r77};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: ret;
+ %res = call {<16 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 5)
+ ret {<16 x i32>, <32 x i32>} %res
+}
+
+define {<8 x i32>, <32 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(i32 %spdesc, <64 x i32> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <32 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call { <8 x i32>, <32 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<106>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x32 {%r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73}, {%r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105}, {%r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r102, %r103, %r104, %r105};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r98, %r99, %r100, %r101};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r94, %r95, %r96, %r97};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r90, %r91, %r92, %r93};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r86, %r87, %r88, %r89};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r82, %r83, %r84, %r85};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r78, %r79, %r80, %r81};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r74, %r75, %r76, %r77};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r70, %r71, %r72, %r73};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: ret;
+ %res = call {<8 x i32>, <32 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v32i32.v64i32(<64 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 5)
+ ret {<8 x i32>, <32 x i32>} %res
+}
+
+define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(i32 %spdesc, <128 x i32> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <64 x i32> } @test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<210>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+496];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+480];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+464];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+448];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+432];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+416];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+400];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+384];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+368];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+352];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+336];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+320];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+304];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+288];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+272];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+256];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r66, %r67, %r68, %r69}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r70, %r71, %r72, %r73}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r74, %r75, %r76, %r77}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r78, %r79, %r80, %r81}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r82, %r83, %r84, %r85}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r86, %r87, %r88, %r89}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r90, %r91, %r92, %r93}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r94, %r95, %r96, %r97}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r98, %r99, %r100, %r101}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r102, %r103, %r104, %r105}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r106, %r107, %r108, %r109}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r110, %r111, %r112, %r113}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r114, %r115, %r116, %r117}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r118, %r119, %r120, %r121}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r122, %r123, %r124, %r125}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r126, %r127, %r128, %r129}, [test_spcompress_elmsz_b8_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spcompress.b8.b2.sp::2:4.x64 {%r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145}, {%r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208, %r209}, {%r126, %r127, %r128, %r129, %r122, %r123, %r124, %r125, %r118, %r119, %r120, %r121, %r114, %r115, %r116, %r117, %r110, %r111, %r112, %r113, %r106, %r107, %r108, %r109, %r102, %r103, %r104, %r105, %r98, %r99, %r100, %r101, %r94, %r95, %r96, %r97, %r90, %r91, %r92, %r93, %r86, %r87, %r88, %r89, %r82, %r83, %r84, %r85, %r78, %r79, %r80, %r81, %r74, %r75, %r76, %r77, %r70, %r71, %r72, %r73, %r66, %r67, %r68, %r69, %r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r206, %r207, %r208, %r209};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r202, %r203, %r204, %r205};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r198, %r199, %r200, %r201};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r194, %r195, %r196, %r197};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r190, %r191, %r192, %r193};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r186, %r187, %r188, %r189};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r182, %r183, %r184, %r185};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r178, %r179, %r180, %r181};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r174, %r175, %r176, %r177};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r170, %r171, %r172, %r173};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r166, %r167, %r168, %r169};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r162, %r163, %r164, %r165};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r158, %r159, %r160, %r161};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r154, %r155, %r156, %r157};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r150, %r151, %r152, %r153};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r146, %r147, %r148, %r149};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r142, %r143, %r144, %r145};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r138, %r139, %r140, %r141};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
+; CHECK-NEXT: ret;
+ %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 2, i32 6)
+ ret {<16 x i32>, <64 x i32>} %res
+}
+
+define {<8 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(i32 %spdesc, <128 x i32> %data) {
+; FORMAT-LABEL: define { <8 x i32>, <64 x i32> } @test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call { <8 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<202>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+496];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+480];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+464];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+448];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+432];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+416];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+400];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+384];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+368];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+352];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+336];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+320];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+304];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+288];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+272];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+256];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r66, %r67, %r68, %r69}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r70, %r71, %r72, %r73}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r74, %r75, %r76, %r77}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r78, %r79, %r80, %r81}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r82, %r83, %r84, %r85}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r86, %r87, %r88, %r89}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r90, %r91, %r92, %r93}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r94, %r95, %r96, %r97}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r98, %r99, %r100, %r101}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r102, %r103, %r104, %r105}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r106, %r107, %r108, %r109}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r110, %r111, %r112, %r113}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r114, %r115, %r116, %r117}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r118, %r119, %r120, %r121}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r122, %r123, %r124, %r125}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r126, %r127, %r128, %r129}, [test_spcompress_elmsz_b16_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spcompress.b16.b2.sp::2:4.x64 {%r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137}, {%r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201}, {%r126, %r127, %r128, %r129, %r122, %r123, %r124, %r125, %r118, %r119, %r120, %r121, %r114, %r115, %r116, %r117, %r110, %r111, %r112, %r113, %r106, %r107, %r108, %r109, %r102, %r103, %r104, %r105, %r98, %r99, %r100, %r101, %r94, %r95, %r96, %r97, %r90, %r91, %r92, %r93, %r86, %r87, %r88, %r89, %r82, %r83, %r84, %r85, %r78, %r79, %r80, %r81, %r74, %r75, %r76, %r77, %r70, %r71, %r72, %r73, %r66, %r67, %r68, %r69, %r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r198, %r199, %r200, %r201};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r194, %r195, %r196, %r197};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r190, %r191, %r192, %r193};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r186, %r187, %r188, %r189};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r182, %r183, %r184, %r185};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r178, %r179, %r180, %r181};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r174, %r175, %r176, %r177};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r170, %r171, %r172, %r173};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r166, %r167, %r168, %r169};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r162, %r163, %r164, %r165};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r158, %r159, %r160, %r161};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r154, %r155, %r156, %r157};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r150, %r151, %r152, %r153};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r146, %r147, %r148, %r149};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r142, %r143, %r144, %r145};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r138, %r139, %r140, %r141};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
+; CHECK-NEXT: ret;
+ %res = call {<8 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v8i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 2, i32 6)
+ ret {<8 x i32>, <64 x i32>} %res
+}
+
+define {<32 x i32>, <64 x i32>} @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(i32 %spdesc, <128 x i32> %data) {
+; FORMAT-LABEL: define { <32 x i32>, <64 x i32> } @test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call { <32 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spcompress_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<226>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+496];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+480];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+464];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+448];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+432];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+416];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+400];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+384];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+368];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+352];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+336];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+320];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+304];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+288];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+272];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+256];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r66, %r67, %r68, %r69}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r70, %r71, %r72, %r73}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r74, %r75, %r76, %r77}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r78, %r79, %r80, %r81}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r82, %r83, %r84, %r85}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r86, %r87, %r88, %r89}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r90, %r91, %r92, %r93}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r94, %r95, %r96, %r97}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r98, %r99, %r100, %r101}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r102, %r103, %r104, %r105}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r106, %r107, %r108, %r109}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r110, %r111, %r112, %r113}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r114, %r115, %r116, %r117}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r118, %r119, %r120, %r121}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r122, %r123, %r124, %r125}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r126, %r127, %r128, %r129}, [test_spcompress_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spcompress.b8.b4.sp::2:4.x64 {%r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161}, {%r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208, %r209, %r210, %r211, %r212, %r213, %r214, %r215, %r216, %r217, %r218, %r219, %r220, %r221, %r222, %r223, %r224, %r225}, {%r126, %r127, %r128, %r129, %r122, %r123, %r124, %r125, %r118, %r119, %r120, %r121, %r114, %r115, %r116, %r117, %r110, %r111, %r112, %r113, %r106, %r107, %r108, %r109, %r102, %r103, %r104, %r105, %r98, %r99, %r100, %r101, %r94, %r95, %r96, %r97, %r90, %r91, %r92, %r93, %r86, %r87, %r88, %r89, %r82, %r83, %r84, %r85, %r78, %r79, %r80, %r81, %r74, %r75, %r76, %r77, %r70, %r71, %r72, %r73, %r66, %r67, %r68, %r69, %r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r222, %r223, %r224, %r225};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r218, %r219, %r220, %r221};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r214, %r215, %r216, %r217};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r210, %r211, %r212, %r213};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r206, %r207, %r208, %r209};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r202, %r203, %r204, %r205};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r198, %r199, %r200, %r201};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r194, %r195, %r196, %r197};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r190, %r191, %r192, %r193};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r186, %r187, %r188, %r189};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r182, %r183, %r184, %r185};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r178, %r179, %r180, %r181};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r174, %r175, %r176, %r177};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r170, %r171, %r172, %r173};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r166, %r167, %r168, %r169};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r162, %r163, %r164, %r165};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r158, %r159, %r160, %r161};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r154, %r155, %r156, %r157};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r150, %r151, %r152, %r153};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r146, %r147, %r148, %r149};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r142, %r143, %r144, %r145};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r138, %r139, %r140, %r141};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
+; CHECK-NEXT: ret;
+ %res = call {<32 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v32i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 8, i32 4, i32 6)
+ ret {<32 x i32>, <64 x i32>} %res
+}
+
+define {<16 x i32>, <64 x i32>} @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(i32 %spdesc, <128 x i32> %data) {
+; FORMAT-LABEL: define { <16 x i32>, <64 x i32> } @test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call { <16 x i32>, <64 x i32> } @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spcompress_elmsz_b16_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<210>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+496];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+480];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r10, %r11, %r12, %r13}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+464];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r14, %r15, %r16, %r17}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+448];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r18, %r19, %r20, %r21}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+432];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r22, %r23, %r24, %r25}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+416];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r26, %r27, %r28, %r29}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+400];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r30, %r31, %r32, %r33}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+384];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r34, %r35, %r36, %r37}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+368];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r38, %r39, %r40, %r41}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+352];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r42, %r43, %r44, %r45}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+336];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r46, %r47, %r48, %r49}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+320];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r50, %r51, %r52, %r53}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+304];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r54, %r55, %r56, %r57}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+288];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r58, %r59, %r60, %r61}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+272];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r62, %r63, %r64, %r65}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+256];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r66, %r67, %r68, %r69}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r70, %r71, %r72, %r73}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r74, %r75, %r76, %r77}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r78, %r79, %r80, %r81}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r82, %r83, %r84, %r85}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r86, %r87, %r88, %r89}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r90, %r91, %r92, %r93}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r94, %r95, %r96, %r97}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r98, %r99, %r100, %r101}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r102, %r103, %r104, %r105}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r106, %r107, %r108, %r109}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r110, %r111, %r112, %r113}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r114, %r115, %r116, %r117}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r118, %r119, %r120, %r121}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r122, %r123, %r124, %r125}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r126, %r127, %r128, %r129}, [test_spcompress_elmsz_b16_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spcompress.b16.b4.sp::2:4.x64 {%r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145}, {%r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208, %r209}, {%r126, %r127, %r128, %r129, %r122, %r123, %r124, %r125, %r118, %r119, %r120, %r121, %r114, %r115, %r116, %r117, %r110, %r111, %r112, %r113, %r106, %r107, %r108, %r109, %r102, %r103, %r104, %r105, %r98, %r99, %r100, %r101, %r94, %r95, %r96, %r97, %r90, %r91, %r92, %r93, %r86, %r87, %r88, %r89, %r82, %r83, %r84, %r85, %r78, %r79, %r80, %r81, %r74, %r75, %r76, %r77, %r70, %r71, %r72, %r73, %r66, %r67, %r68, %r69, %r62, %r63, %r64, %r65, %r58, %r59, %r60, %r61, %r54, %r55, %r56, %r57, %r50, %r51, %r52, %r53, %r46, %r47, %r48, %r49, %r42, %r43, %r44, %r45, %r38, %r39, %r40, %r41, %r34, %r35, %r36, %r37, %r30, %r31, %r32, %r33, %r26, %r27, %r28, %r29, %r22, %r23, %r24, %r25, %r18, %r19, %r20, %r21, %r14, %r15, %r16, %r17, %r10, %r11, %r12, %r13, %r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5}, %r1;
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r206, %r207, %r208, %r209};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r202, %r203, %r204, %r205};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r198, %r199, %r200, %r201};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r194, %r195, %r196, %r197};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r190, %r191, %r192, %r193};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r186, %r187, %r188, %r189};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r182, %r183, %r184, %r185};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r178, %r179, %r180, %r181};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r174, %r175, %r176, %r177};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r170, %r171, %r172, %r173};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r166, %r167, %r168, %r169};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r162, %r163, %r164, %r165};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r158, %r159, %r160, %r161};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r154, %r155, %r156, %r157};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r150, %r151, %r152, %r153};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r146, %r147, %r148, %r149};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r142, %r143, %r144, %r145};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r138, %r139, %r140, %r141};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r134, %r135, %r136, %r137};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r130, %r131, %r132, %r133};
+; CHECK-NEXT: ret;
+ %res = call {<16 x i32>, <64 x i32>} @llvm.nvvm.spcompress.sp2to4.v16i32.v64i32.v128i32(<128 x i32> %data, i32 %spdesc, i32 16, i32 4, i32 6)
+ ret {<16 x i32>, <64 x i32>} %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
new file mode 100644
index 0000000000000..74d6eaa53dc66
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-invalid.ll
@@ -0,0 +1,105 @@
+; RUN: split-file %s %t
+; RUN: not llvm-as -disable-output < %t/bad-elem-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-ELEM
+; RUN: not llvm-as -disable-output < %t/bad-idx-size.ll 2>&1 | FileCheck %s --check-prefix=BAD-IDX
+; RUN: not llvm-as -disable-output < %t/bad-repeat-factor.ll 2>&1 | FileCheck %s --check-prefix=BAD-RPF
+; RUN: not --crash llc < %t/bad-idx-size-for-target.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-IDX-TARGET
+; RUN: not --crash llc < %t/bad-input-span.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-INPUT-SPAN
+; RUN: not --crash llc < %t/bad-data-too-small.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA-SMALL
+; RUN: not --crash llc < %t/bad-mdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA
+; RUN: not --crash llc < %t/bad-cdata-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-CDATA
+; RUN: not --crash llc < %t/bad-data-size.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-DATA
+; RUN: not --crash llc < %t/bad-result-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-RESULT-TY
+; RUN: not --crash llc < %t/bad-mdata-type.ll -march=nvptx64 -mcpu=sm_107a 2>&1 | FileCheck %s --check-prefix=BAD-MDATA-TY
+
+;--- bad-elem-size.ll
+
+define i32 @bad_spdecompress_elem_size(i32 %metadata, i32 %compdata) {
+; BAD-ELEM: immarg value 9 for arg 2 out of range set
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 9, i32 2, i32 0)
+ ret i32 %res
+}
+
+;--- bad-idx-size.ll
+
+define i32 @bad_spdecompress_idx_size(i32 %metadata, i32 %compdata) {
+; BAD-IDX: immarg value 3 for arg 3 out of range set
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 3, i32 0)
+ ret i32 %res
+}
+
+;--- bad-repeat-factor.ll
+
+define i32 @bad_spdecompress_repeat_factor(i32 %metadata, i32 %compdata) {
+; BAD-RPF: immarg value 7 for arg 4 out of range [0,7)
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 7)
+ ret i32 %res
+}
+
+;--- bad-idx-size-for-target.ll
+
+define <8 x i32> @bad_spdecompress_idx_size_for_target(
+ i32 %metadata, i32 %compdata) {
+; BAD-IDX-TARGET: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=8, elem_size=8, idx_size=2, repeat_factor=2
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+ ret <8 x i32> %res
+}
+
+;--- bad-input-span.ll
+
+define <4 x i32> @bad_spdecompress_input_span(<2 x i32> %metadata,
+ <2 x i32> %compdata) {
+; BAD-INPUT-SPAN: LLVM ERROR: Invalid spdecompress flags: num_src=4, num_tgt=8, elem_size=16, idx_size=4, repeat_factor=0
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+;--- bad-data-too-small.ll
+
+define i32 @bad_spdecompress_data_too_small(i32 %metadata, i32 %compdata) {
+; BAD-DATA-SMALL: LLVM ERROR: Invalid spdecompress flags: num_src=1, num_tgt=2, elem_size=8, idx_size=2, repeat_factor=0
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i32 %res
+}
+
+;--- bad-mdata-size.ll
+
+define <4 x i32> @bad_spdecompress_metadata_register_count(
+ <2 x i32> %metadata, <2 x i32> %compdata) {
+; BAD-MDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.v2i32.v2i32(<2 x i32> %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <4 x i32> %res
+}
+
+;--- bad-cdata-size.ll
+
+define <4 x i32> @bad_spdecompress_compressed_data_register_count(
+ i32 %metadata, i32 %compdata) {
+; BAD-CDATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+;--- bad-data-size.ll
+
+define <2 x i32> @bad_spdecompress_data_register_count(
+ i32 %metadata, <2 x i32> %compdata) {
+; BAD-DATA: LLVM ERROR: spdecompress operand/result types do not match the SP intrinsic flags
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <2 x i32> %res
+}
+
+;--- bad-result-type.ll
+
+define i64 @bad_spdecompress_result_type(i32 %metadata, i32 %compdata) {
+; BAD-RESULT-TY: LLVM ERROR: spdecompress expects scalarized i32 results
+ %res = call i64 @llvm.nvvm.spdecompress.sp2to4.i64.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i64 %res
+}
+
+;--- bad-mdata-type.ll
+
+define i32 @bad_spdecompress_metadata_type(i64 %metadata, i32 %compdata) {
+; BAD-MDATA-TY: LLVM ERROR: spdecompress expects scalarized i32 operands
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i64.i32(i64 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i32 %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
new file mode 100644
index 0000000000000..e375269b36e49
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to16.ll
@@ -0,0 +1,309 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4
+define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<19>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x4 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<36>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x8 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r32, %r33, %r34, %r35};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r28, %r29, %r30, %r31};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r24, %r25, %r26, %r27};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r20, %r21, %r22, %r23};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<71>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x16 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<141>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to16_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:16.x32 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4
+define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x1 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<19>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to16.v16i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<36>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r32, %r33, %r34, %r35};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r28, %r29, %r30, %r31};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r24, %r25, %r26, %r27};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r20, %r21, %r22, %r23};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to16.v32i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<70>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r66, %r67, %r68, %r69};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r62, %r63, %r64, %r65};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r58, %r59, %r60, %r61};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r54, %r55, %r56, %r57};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r50, %r51, %r52, %r53};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r46, %r47, %r48, %r49};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r42, %r43, %r44, %r45};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r30, %r31, %r32, %r33};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to16.v64i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<139>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to16_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:16.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r135, %r136, %r137, %r138};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r131, %r132, %r133, %r134};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r127, %r128, %r129, %r130};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r123, %r124, %r125, %r126};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r119, %r120, %r121, %r122};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r115, %r116, %r117, %r118};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r111, %r112, %r113, %r114};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r107, %r108, %r109, %r110};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r103, %r104, %r105, %r106};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r99, %r100, %r101, %r102};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r95, %r96, %r97, %r98};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r91, %r92, %r93, %r94};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r87, %r88, %r89, %r90};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r83, %r84, %r85, %r86};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r79, %r80, %r81, %r82};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r75, %r76, %r77, %r78};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r71, %r72, %r73, %r74};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to16.v128i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
new file mode 100644
index 0000000000000..e90aed6e35fb0
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to2.ll
@@ -0,0 +1,552 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x2 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 3)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x16 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 4)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x32 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 5)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<53>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:2.x64 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 6)
+ ret <32 x i32> %res
+}
+
+define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x2 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x4 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x8 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x16 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<29>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x32 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<57>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to2_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:2.x64 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <32 x i32> %res
+}
+
+define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 1)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 2)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 3)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<26>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x16 {%r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25}, {%r1}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 4)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<51>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r11, %r12, %r13, %r14}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r15, %r16, %r17, %r18}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x32 {%r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50}, {%r1, %r2}, {%r15, %r16, %r17, %r18, %r11, %r12, %r13, %r14, %r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 5)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<101>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:2.x64 {%r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100}, {%r1, %r2, %r3, %r4}, {%r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 6)
+ ret <64 x i32> %res
+}
+
+define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to2.i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to2.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to2.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to2.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to2.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<53>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x32 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to2.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<105>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp1to2_elmsz_b16_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:2.x64 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to2.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 6)
+ ret <64 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
new file mode 100644
index 0000000000000..c9c369e724a07
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to4.ll
@@ -0,0 +1,682 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=2
+define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 2)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x8 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 3)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<22>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x16 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 4)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<43>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x32 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 5)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<85>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::1:4.x64 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 6)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4
+define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp1to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret i32 %res
+}
+
+define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x4 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x8 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<23>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x16 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<45>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x32 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<89>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to4_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:4.x64 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=2
+define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 1)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 2)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<22>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 3)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(i32 %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<42>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x16 {%r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41}, {%r1}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r38, %r39, %r40, %r41};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r30, %r31, %r32, %r33};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 4)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(<2 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<83>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r11, %r12, %r13, %r14}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r15, %r16, %r17, %r18}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x32 {%r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82}, {%r1, %r2}, {%r15, %r16, %r17, %r18, %r11, %r12, %r13, %r14, %r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r79, %r80, %r81, %r82};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r75, %r76, %r77, %r78};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r71, %r72, %r73, %r74};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 5)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(<4 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<165>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::1:4.x64 {%r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164}, {%r1, %r2, %r3, %r4}, {%r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 6)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4
+define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to4.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to4.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<22>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to4.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<43>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to4.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<85>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x32 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to4.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<169>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp1to4_elmsz_b16_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:4.x64 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to4.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 6)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
new file mode 100644
index 0000000000000..c9c5347fa485e
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp1to8.ll
@@ -0,0 +1,350 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4
+define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp1to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x4 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<20>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x8 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<39>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x16 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<77>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x32 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<153>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp1to8_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::1:8.x64 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4
+define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp1to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp1to8.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<20>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp1to8.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<38>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r30, %r31, %r32, %r33};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp1to8.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<75>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r71, %r72, %r73, %r74};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp1to8.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<149>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp1to8_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::1:8.x32 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp1to8.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
new file mode 100644
index 0000000000000..a85bda8ebb5e4
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to16.ll
@@ -0,0 +1,325 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4, rpf=1
+define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x2 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<20>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<39>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x8 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<77>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x16 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<153>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to16_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:16.x32 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=1
+define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<11>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x1 {%r3, %r4, %r5, %r6, %r7, %r8, %r9, %r10}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to16.v8i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <8 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=2
+define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<20>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x2 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r16, %r17, %r18, %r19};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r12, %r13, %r14, %r15};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to16.v16i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ ret <16 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=4
+define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<38>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x4 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r34, %r35, %r36, %r37};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r30, %r31, %r32, %r33};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r26, %r27, %r28, %r29};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to16.v32i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <32 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=8
+define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<75>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x8 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r71, %r72, %r73, %r74};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r67, %r68, %r69, %r70};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r63, %r64, %r65, %r66};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r59, %r60, %r61, %r62};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r55, %r56, %r57, %r58};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r51, %r52, %r53, %r54};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to16.v64i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=16
+define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<149>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to16_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:16.x16 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to16.v128i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
new file mode 100644
index 0000000000000..d332a2e5fc790
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to4.ll
@@ -0,0 +1,758 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=2, rpf=1
+define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 0)
+ ret i32 %res
+}
+
+; elem_size=8, idx_size=2, rpf=2
+define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 2, i32 1)
+ ret <2 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=4
+define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 2, i32 2)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=8
+define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x8 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 8, i32 2, i32 3)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=16
+define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x16 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 2, i32 4)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=32
+define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<53>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x32 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 2, i32 5)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=2, rpf=64
+define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<105>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b2.sp::2:4.x64 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 2, i32 6)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=1
+define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define i32 @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<4>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x1 {%r3}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.b32 [func_retval0], %r3;
+; CHECK-NEXT: ret;
+ %res = call i32 @llvm.nvvm.spdecompress.sp2to4.i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret i32 %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x2 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <2 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x4 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x8 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<29>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x16 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<57>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x32 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=64
+define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<113>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp2to4_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:4.x64 {%r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=1
+define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 2, i32 0)
+ ret <2 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=2
+define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 2, i32 1)
+ ret <4 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=4
+define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x4 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 2, i32 2)
+ ret <8 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=8
+define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(i32 %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<26>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r6, %r7, %r8, %r9}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x8 {%r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25}, {%r1}, {%r6, %r7, %r8, %r9, %r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r22, %r23, %r24, %r25};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.i32.v8i32(i32 %metadata, <8 x i32> %compdata, i32 16, i32 2, i32 3)
+ ret <16 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=16
+define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(<2 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<51>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r11, %r12, %r13, %r14}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r15, %r16, %r17, %r18}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x16 {%r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50}, {%r1, %r2}, {%r15, %r16, %r17, %r18, %r11, %r12, %r13, %r14, %r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r47, %r48, %r49, %r50};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r43, %r44, %r45, %r46};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v2i32.v16i32(<2 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 2, i32 4)
+ ret <32 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=32
+define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(<4 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<101>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x32 {%r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100}, {%r1, %r2, %r3, %r4}, {%r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v4i32.v32i32(<4 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 2, i32 5)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=2, rpf=64
+define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(<8 x i32> %metadata, <64 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b2 */ i32 2, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<201>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r49, %r50, %r51, %r52}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r53, %r54, %r55, %r56}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r57, %r58, %r59, %r60}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r61, %r62, %r63, %r64}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r65, %r66, %r67, %r68}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r69, %r70, %r71, %r72}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b2_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b2.sp::2:4.x64 {%r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r69, %r70, %r71, %r72, %r65, %r66, %r67, %r68, %r61, %r62, %r63, %r64, %r57, %r58, %r59, %r60, %r53, %r54, %r55, %r56, %r49, %r50, %r51, %r52, %r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r197, %r198, %r199, %r200};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r193, %r194, %r195, %r196};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r189, %r190, %r191, %r192};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r185, %r186, %r187, %r188};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r181, %r182, %r183, %r184};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r177, %r178, %r179, %r180};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v8i32.v64i32(<8 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 2, i32 6)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=1
+define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to4.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=2
+define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to4.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=4
+define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<14>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x4 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to4.v8i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=8
+define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<27>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x8 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to4.v16i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=16
+define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<53>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x16 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to4.v32i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=32
+define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<105>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x32 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to4.v64i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=64
+define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(<16 x i32> %metadata, <64 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<209>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r49, %r50, %r51, %r52}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r53, %r54, %r55, %r56}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r57, %r58, %r59, %r60}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r61, %r62, %r63, %r64}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r65, %r66, %r67, %r68}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r69, %r70, %r71, %r72}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r73, %r74, %r75, %r76}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r77, %r78, %r79, %r80}, [test_spdecompress_sp2to4_elmsz_b16_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:4.x64 {%r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r77, %r78, %r79, %r80, %r73, %r74, %r75, %r76, %r69, %r70, %r71, %r72, %r65, %r66, %r67, %r68, %r61, %r62, %r63, %r64, %r57, %r58, %r59, %r60, %r53, %r54, %r55, %r56, %r49, %r50, %r51, %r52, %r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r205, %r206, %r207, %r208};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r201, %r202, %r203, %r204};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r197, %r198, %r199, %r200};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r193, %r194, %r195, %r196};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r189, %r190, %r191, %r192};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r185, %r186, %r187, %r188};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r181, %r182, %r183, %r184};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r177, %r178, %r179, %r180};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to4.v128i32.v16i32.v64i32(<16 x i32> %metadata, <64 x i32> %compdata, i32 16, i32 4, i32 6)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
new file mode 100644
index 0000000000000..452efc202134f
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp2to8.ll
@@ -0,0 +1,379 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4, rpf=1
+define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp2to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x2 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x4 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<23>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x8 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<45>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x16 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<89>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x32 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=64
+define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(<16 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<177>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp2to8_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::2:8.x64 {%r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <128 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=1
+define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp2to8.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 16, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=2
+define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x2 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp2to8.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 16, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=4
+define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(i32 %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<22>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r2, %r3, %r4, %r5}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x4 {%r6, %r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21}, {%r1}, {%r2, %r3, %r4, %r5};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r18, %r19, %r20, %r21};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r14, %r15, %r16, %r17};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r10, %r11, %r12, %r13};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r6, %r7, %r8, %r9};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp2to8.v16i32.i32.v4i32(i32 %metadata, <4 x i32> %compdata, i32 16, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=8
+define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(<2 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<43>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r7, %r8, %r9, %r10}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x8 {%r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42}, {%r1, %r2}, {%r7, %r8, %r9, %r10, %r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r39, %r40, %r41, %r42};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r35, %r36, %r37, %r38};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r31, %r32, %r33, %r34};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r27, %r28, %r29, %r30};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r23, %r24, %r25, %r26};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp2to8.v32i32.v2i32.v8i32(<2 x i32> %metadata, <8 x i32> %compdata, i32 16, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=16
+define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(<4 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<85>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x16 {%r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84}, {%r1, %r2, %r3, %r4}, {%r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp2to8.v64i32.v4i32.v16i32(<4 x i32> %metadata, <16 x i32> %compdata, i32 16, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+; elem_size=16, idx_size=4, rpf=32
+define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(<8 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b16 */ i32 16, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<169>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp2to8_elmsz_b16_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b16.b4.sp::2:8.x32 {%r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp2to8.v128i32.v8i32.v32i32(<8 x i32> %metadata, <32 x i32> %compdata, i32 16, i32 4, i32 5)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
new file mode 100644
index 0000000000000..ad2f926a52085
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to16.ll
@@ -0,0 +1,184 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4, rpf=1
+define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<7>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x1 {%r3, %r4, %r5, %r6}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to16.v4i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<12>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x2 {%r4, %r5, %r6, %r7, %r8, %r9, %r10, %r11}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r8, %r9, %r10, %r11};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to16.v8i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 1)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<23>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x4 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r19, %r20, %r21, %r22};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r15, %r16, %r17, %r18};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to16.v16i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<45>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x8 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to16.v32i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<89>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x16 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to16.v64i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<177>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp4to16_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:16.x32 {%r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to16.v128i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <128 x i32> %res
+}
diff --git a/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
new file mode 100644
index 0000000000000..94bff5c02f6f2
--- /dev/null
+++ b/llvm/test/CodeGen/NVPTX/spdecompress-sp4to8.ll
@@ -0,0 +1,224 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s -march=nvptx64 -mcpu=sm_107a | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis | FileCheck %s --check-prefixes=FORMAT
+; RUN: %if ptxas-sm_107a %{ llc < %s -march=nvptx64 -mcpu=sm_107a | %ptxas-verify -arch=sm_107a %}
+
+; elem_size=8, idx_size=4, rpf=1
+define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(i32 %metadata, i32 %compdata) {
+; FORMAT-LABEL: define <2 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
+; FORMAT-NEXT: %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x1 */ i32 0)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<5>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1_param_0];
+; CHECK-NEXT: ld.param::func.b32 %r2, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_1_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x1 {%r3, %r4}, {%r1}, {%r2};
+; CHECK-NEXT: st.param::func.v2.b32 [func_retval0], {%r3, %r4};
+; CHECK-NEXT: ret;
+ %res = call <2 x i32> @llvm.nvvm.spdecompress.sp4to8.v2i32.i32.i32(i32 %metadata, i32 %compdata, i32 8, i32 4, i32 0)
+ ret <2 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=2
+define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(i32 %metadata, <2 x i32> %compdata) {
+; FORMAT-LABEL: define <4 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
+; FORMAT-NEXT: %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x2 */ i32 1)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<8>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.b32 %r1, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2_param_0];
+; CHECK-NEXT: ld.param::func.v2.b32 {%r2, %r3}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_2_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x2 {%r4, %r5, %r6, %r7}, {%r1}, {%r2, %r3};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r4, %r5, %r6, %r7};
+; CHECK-NEXT: ret;
+ %res = call <4 x i32> @llvm.nvvm.spdecompress.sp4to8.v4i32.i32.v2i32(i32 %metadata, <2 x i32> %compdata, i32 8, i32 4, i32 1)
+ ret <4 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=4
+define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(<2 x i32> %metadata, <4 x i32> %compdata) {
+; FORMAT-LABEL: define <8 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
+; FORMAT-NEXT: %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x4 */ i32 2)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<15>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v2.b32 {%r1, %r2}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r3, %r4, %r5, %r6}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_4_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x4 {%r7, %r8, %r9, %r10, %r11, %r12, %r13, %r14}, {%r1, %r2}, {%r3, %r4, %r5, %r6};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r11, %r12, %r13, %r14};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r7, %r8, %r9, %r10};
+; CHECK-NEXT: ret;
+ %res = call <8 x i32> @llvm.nvvm.spdecompress.sp4to8.v8i32.v2i32.v4i32(<2 x i32> %metadata, <4 x i32> %compdata, i32 8, i32 4, i32 2)
+ ret <8 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=8
+define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(<4 x i32> %metadata, <8 x i32> %compdata) {
+; FORMAT-LABEL: define <16 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
+; FORMAT-NEXT: %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x8 */ i32 3)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<29>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_8_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x8 {%r13, %r14, %r15, %r16, %r17, %r18, %r19, %r20, %r21, %r22, %r23, %r24, %r25, %r26, %r27, %r28}, {%r1, %r2, %r3, %r4}, {%r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r21, %r22, %r23, %r24};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r13, %r14, %r15, %r16};
+; CHECK-NEXT: ret;
+ %res = call <16 x i32> @llvm.nvvm.spdecompress.sp4to8.v16i32.v4i32.v8i32(<4 x i32> %metadata, <8 x i32> %compdata, i32 8, i32 4, i32 3)
+ ret <16 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=16
+define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(<8 x i32> %metadata, <16 x i32> %compdata) {
+; FORMAT-LABEL: define <32 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
+; FORMAT-NEXT: %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x16 */ i32 4)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<57>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_16_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x16 {%r25, %r26, %r27, %r28, %r29, %r30, %r31, %r32, %r33, %r34, %r35, %r36, %r37, %r38, %r39, %r40, %r41, %r42, %r43, %r44, %r45, %r46, %r47, %r48, %r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56}, {%r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r45, %r46, %r47, %r48};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r41, %r42, %r43, %r44};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r37, %r38, %r39, %r40};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r29, %r30, %r31, %r32};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r25, %r26, %r27, %r28};
+; CHECK-NEXT: ret;
+ %res = call <32 x i32> @llvm.nvvm.spdecompress.sp4to8.v32i32.v8i32.v16i32(<8 x i32> %metadata, <16 x i32> %compdata, i32 8, i32 4, i32 4)
+ ret <32 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=32
+define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(<16 x i32> %metadata, <32 x i32> %compdata) {
+; FORMAT-LABEL: define <64 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
+; FORMAT-NEXT: %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x32 */ i32 5)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<113>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_32_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x32 {%r49, %r50, %r51, %r52, %r53, %r54, %r55, %r56, %r57, %r58, %r59, %r60, %r61, %r62, %r63, %r64, %r65, %r66, %r67, %r68, %r69, %r70, %r71, %r72, %r73, %r74, %r75, %r76, %r77, %r78, %r79, %r80, %r81, %r82, %r83, %r84, %r85, %r86, %r87, %r88, %r89, %r90, %r91, %r92, %r93, %r94, %r95, %r96, %r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112}, {%r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36, %r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r93, %r94, %r95, %r96};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r89, %r90, %r91, %r92};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r85, %r86, %r87, %r88};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r81, %r82, %r83, %r84};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r77, %r78, %r79, %r80};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r73, %r74, %r75, %r76};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r69, %r70, %r71, %r72};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r65, %r66, %r67, %r68};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r61, %r62, %r63, %r64};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r57, %r58, %r59, %r60};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r53, %r54, %r55, %r56};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r49, %r50, %r51, %r52};
+; CHECK-NEXT: ret;
+ %res = call <64 x i32> @llvm.nvvm.spdecompress.sp4to8.v64i32.v16i32.v32i32(<16 x i32> %metadata, <32 x i32> %compdata, i32 8, i32 4, i32 5)
+ ret <64 x i32> %res
+}
+
+; elem_size=8, idx_size=4, rpf=64
+define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(<32 x i32> %metadata, <64 x i32> %compdata) {
+; FORMAT-LABEL: define <128 x i32> @test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
+; FORMAT-NEXT: %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, /* elem_size=.b8 */ i32 8, /* idx_size=.b4 */ i32 4, /* repeat_factor=.x64 */ i32 6)
+; CHECK-LABEL: test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64(
+; CHECK: {
+; CHECK-NEXT: .reg .b32 %r<225>;
+; CHECK-EMPTY:
+; CHECK-NEXT: // %bb.0:
+; CHECK-NEXT: ld.param::func.v4.b32 {%r1, %r2, %r3, %r4}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r5, %r6, %r7, %r8}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r9, %r10, %r11, %r12}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r13, %r14, %r15, %r16}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r17, %r18, %r19, %r20}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r21, %r22, %r23, %r24}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r25, %r26, %r27, %r28}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r29, %r30, %r31, %r32}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_0];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r33, %r34, %r35, %r36}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+240];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r37, %r38, %r39, %r40}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+224];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r41, %r42, %r43, %r44}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+208];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r45, %r46, %r47, %r48}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+192];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r49, %r50, %r51, %r52}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+176];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r53, %r54, %r55, %r56}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+160];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r57, %r58, %r59, %r60}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+144];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r61, %r62, %r63, %r64}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+128];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r65, %r66, %r67, %r68}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+112];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r69, %r70, %r71, %r72}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+96];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r73, %r74, %r75, %r76}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+80];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r77, %r78, %r79, %r80}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+64];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r81, %r82, %r83, %r84}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+48];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r85, %r86, %r87, %r88}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+32];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r89, %r90, %r91, %r92}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1+16];
+; CHECK-NEXT: ld.param::func.v4.b32 {%r93, %r94, %r95, %r96}, [test_spdecompress_sp4to8_elmsz_b8_idxsz_b4_rpf_64_param_1];
+; CHECK-NEXT: spdecompress.b8.b4.sp::4:8.x64 {%r97, %r98, %r99, %r100, %r101, %r102, %r103, %r104, %r105, %r106, %r107, %r108, %r109, %r110, %r111, %r112, %r113, %r114, %r115, %r116, %r117, %r118, %r119, %r120, %r121, %r122, %r123, %r124, %r125, %r126, %r127, %r128, %r129, %r130, %r131, %r132, %r133, %r134, %r135, %r136, %r137, %r138, %r139, %r140, %r141, %r142, %r143, %r144, %r145, %r146, %r147, %r148, %r149, %r150, %r151, %r152, %r153, %r154, %r155, %r156, %r157, %r158, %r159, %r160, %r161, %r162, %r163, %r164, %r165, %r166, %r167, %r168, %r169, %r170, %r171, %r172, %r173, %r174, %r175, %r176, %r177, %r178, %r179, %r180, %r181, %r182, %r183, %r184, %r185, %r186, %r187, %r188, %r189, %r190, %r191, %r192, %r193, %r194, %r195, %r196, %r197, %r198, %r199, %r200, %r201, %r202, %r203, %r204, %r205, %r206, %r207, %r208, %r209, %r210, %r211, %r212, %r213, %r214, %r215, %r216, %r217, %r218, %r219, %r220, %r221, %r222, %r223, %r224}, {%r29, %r30, %r31, %r32, %r25, %r26, %r27, %r28, %r21, %r22, %r23, %r24, %r17, %r18, %r19, %r20, %r13, %r14, %r15, %r16, %r9, %r10, %r11, %r12, %r5, %r6, %r7, %r8, %r1, %r2, %r3, %r4}, {%r93, %r94, %r95, %r96, %r89, %r90, %r91, %r92, %r85, %r86, %r87, %r88, %r81, %r82, %r83, %r84, %r77, %r78, %r79, %r80, %r73, %r74, %r75, %r76, %r69, %r70, %r71, %r72, %r65, %r66, %r67, %r68, %r61, %r62, %r63, %r64, %r57, %r58, %r59, %r60, %r53, %r54, %r55, %r56, %r49, %r50, %r51, %r52, %r45, %r46, %r47, %r48, %r41, %r42, %r43, %r44, %r37, %r38, %r39, %r40, %r33, %r34, %r35, %r36};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+496], {%r221, %r222, %r223, %r224};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+480], {%r217, %r218, %r219, %r220};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+464], {%r213, %r214, %r215, %r216};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+448], {%r209, %r210, %r211, %r212};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+432], {%r205, %r206, %r207, %r208};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+416], {%r201, %r202, %r203, %r204};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+400], {%r197, %r198, %r199, %r200};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+384], {%r193, %r194, %r195, %r196};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+368], {%r189, %r190, %r191, %r192};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+352], {%r185, %r186, %r187, %r188};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+336], {%r181, %r182, %r183, %r184};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+320], {%r177, %r178, %r179, %r180};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+304], {%r173, %r174, %r175, %r176};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+288], {%r169, %r170, %r171, %r172};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+272], {%r165, %r166, %r167, %r168};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+256], {%r161, %r162, %r163, %r164};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+240], {%r157, %r158, %r159, %r160};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+224], {%r153, %r154, %r155, %r156};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+208], {%r149, %r150, %r151, %r152};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+192], {%r145, %r146, %r147, %r148};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+176], {%r141, %r142, %r143, %r144};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+160], {%r137, %r138, %r139, %r140};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+144], {%r133, %r134, %r135, %r136};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+128], {%r129, %r130, %r131, %r132};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+112], {%r125, %r126, %r127, %r128};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+96], {%r121, %r122, %r123, %r124};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+80], {%r117, %r118, %r119, %r120};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+64], {%r113, %r114, %r115, %r116};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+48], {%r109, %r110, %r111, %r112};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+32], {%r105, %r106, %r107, %r108};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0+16], {%r101, %r102, %r103, %r104};
+; CHECK-NEXT: st.param::func.v4.b32 [func_retval0], {%r97, %r98, %r99, %r100};
+; CHECK-NEXT: ret;
+ %res = call <128 x i32> @llvm.nvvm.spdecompress.sp4to8.v128i32.v32i32.v64i32(<32 x i32> %metadata, <64 x i32> %compdata, i32 8, i32 4, i32 6)
+ ret <128 x i32> %res
+}
More information about the llvm-commits
mailing list