[llvm] r361953 - [SVE][IR] Scalable Vector IR Type
Graham Hunter via llvm-commits
llvm-commits at lists.llvm.org
Wed May 29 05:22:54 PDT 2019
Author: huntergr
Date: Wed May 29 05:22:54 2019
New Revision: 361953
URL: http://llvm.org/viewvc/llvm-project?rev=361953&view=rev
Log:
[SVE][IR] Scalable Vector IR Type
* Adds a 'scalable' flag to VectorType
* Adds an 'ElementCount' class to VectorType to pass (possibly scalable) vector lengths, with overloaded operators.
* Modifies existing helper functions to use ElementCount
* Adds support for serializing/deserializing to/from both textual and bitcode IR formats
* Extends the verifier to reject global variables of scalable types
* Updates documentation
See the latest version of the RFC here: http://lists.llvm.org/pipermail/llvm-dev/2018-July/124396.html
Reviewers: rengolin, lattner, echristo, chandlerc, hfinkel, rkruppe, samparker, SjoerdMeijer, greened, sebpop
Reviewed By: hfinkel, sebpop
Differential Revision: https://reviews.llvm.org/D32530
Added:
llvm/trunk/include/llvm/Support/ScalableSize.h
llvm/trunk/test/Verifier/scalable-aggregates.ll
llvm/trunk/test/Verifier/scalable-global-vars.ll
llvm/trunk/unittests/IR/VectorTypesTest.cpp
Modified:
llvm/trunk/docs/LangRef.rst
llvm/trunk/include/llvm/ADT/DenseMapInfo.h
llvm/trunk/include/llvm/IR/DerivedTypes.h
llvm/trunk/include/llvm/IR/Type.h
llvm/trunk/lib/AsmParser/LLLexer.cpp
llvm/trunk/lib/AsmParser/LLParser.cpp
llvm/trunk/lib/AsmParser/LLToken.h
llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp
llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp
llvm/trunk/lib/IR/AsmWriter.cpp
llvm/trunk/lib/IR/LLVMContextImpl.h
llvm/trunk/lib/IR/Type.cpp
llvm/trunk/lib/IR/Verifier.cpp
llvm/trunk/test/Bitcode/compatibility.ll
llvm/trunk/unittests/IR/CMakeLists.txt
Modified: llvm/trunk/docs/LangRef.rst
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/LangRef.rst?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/docs/LangRef.rst (original)
+++ llvm/trunk/docs/LangRef.rst Wed May 29 05:22:54 2019
@@ -675,6 +675,9 @@ an optional list of attached :ref:`metad
Variables and aliases can have a
:ref:`Thread Local Storage Model <tls_model>`.
+:ref:`Scalable vectors <t_vector>` cannot be global variables or members of
+structs or arrays because their size is unknown at compile time.
+
Syntax::
@<GlobalVarName> = [Linkage] [PreemptionSpecifier] [Visibility]
@@ -2730,30 +2733,40 @@ Vector Type
A vector type is a simple derived type that represents a vector of
elements. Vector types are used when multiple primitive data are
operated in parallel using a single instruction (SIMD). A vector type
-requires a size (number of elements) and an underlying primitive data
-type. Vector types are considered :ref:`first class <t_firstclass>`.
+requires a size (number of elements), an underlying primitive data type,
+and a scalable property to represent vectors where the exact hardware
+vector length is unknown at compile time. Vector types are considered
+:ref:`first class <t_firstclass>`.
:Syntax:
::
- < <# elements> x <elementtype> >
+ < <# elements> x <elementtype> > ; Fixed-length vector
+ < vscale x <# elements> x <elementtype> > ; Scalable vector
The number of elements is a constant integer value larger than 0;
elementtype may be any integer, floating-point or pointer type. Vectors
-of size zero are not allowed.
+of size zero are not allowed. For scalable vectors, the total number of
+elements is a constant multiple (called vscale) of the specified number
+of elements; vscale is a positive integer that is unknown at compile time
+and the same hardware-dependent constant for all scalable vectors at run
+time. The size of a specific scalable vector type is thus constant within
+IR, even if the exact size in bytes cannot be determined until run time.
:Examples:
-+-------------------+--------------------------------------------------+
-| ``<4 x i32>`` | Vector of 4 32-bit integer values. |
-+-------------------+--------------------------------------------------+
-| ``<8 x float>`` | Vector of 8 32-bit floating-point values. |
-+-------------------+--------------------------------------------------+
-| ``<2 x i64>`` | Vector of 2 64-bit integer values. |
-+-------------------+--------------------------------------------------+
-| ``<4 x i64*>`` | Vector of 4 pointers to 64-bit integer values. |
-+-------------------+--------------------------------------------------+
++------------------------+----------------------------------------------------+
+| ``<4 x i32>`` | Vector of 4 32-bit integer values. |
++------------------------+----------------------------------------------------+
+| ``<8 x float>`` | Vector of 8 32-bit floating-point values. |
++------------------------+----------------------------------------------------+
+| ``<2 x i64>`` | Vector of 2 64-bit integer values. |
++------------------------+----------------------------------------------------+
+| ``<4 x i64*>`` | Vector of 4 pointers to 64-bit integer values. |
++------------------------+----------------------------------------------------+
+| ``<vscale x 4 x i32>`` | Vector with a multiple of 4 32-bit integer values. |
++------------------------+----------------------------------------------------+
.. _t_label:
@@ -8135,6 +8148,7 @@ Syntax:
::
<result> = extractelement <n x <ty>> <val>, <ty2> <idx> ; yields <ty>
+ <result> = extractelement <vscale x n x <ty>> <val>, <ty2> <idx> ; yields <ty>
Overview:
"""""""""
@@ -8155,7 +8169,9 @@ Semantics:
The result is a scalar of the same type as the element type of ``val``.
Its value is the value at position ``idx`` of ``val``. If ``idx``
-exceeds the length of ``val``, the result is a
+exceeds the length of ``val`` for a fixed-length vector, the result is a
+:ref:`poison value <poisonvalues>`. For a scalable vector, if the value
+of ``idx`` exceeds the runtime length of the vector, the result is a
:ref:`poison value <poisonvalues>`.
Example:
@@ -8176,6 +8192,7 @@ Syntax:
::
<result> = insertelement <n x <ty>> <val>, <ty> <elt>, <ty2> <idx> ; yields <n x <ty>>
+ <result> = insertelement <vscale x n x <ty>> <val>, <ty> <elt>, <ty2> <idx> ; yields <vscale x n x <ty>>
Overview:
"""""""""
@@ -8197,7 +8214,9 @@ Semantics:
The result is a vector of the same type as ``val``. Its element values
are those of ``val`` except at position ``idx``, where it gets the value
-``elt``. If ``idx`` exceeds the length of ``val``, the result
+``elt``. If ``idx`` exceeds the length of ``val`` for a fixed-length vector,
+the result is a :ref:`poison value <poisonvalues>`. For a scalable vector,
+if the value of ``idx`` exceeds the runtime length of the vector, the result
is a :ref:`poison value <poisonvalues>`.
Example:
@@ -8218,6 +8237,7 @@ Syntax:
::
<result> = shufflevector <n x <ty>> <v1>, <n x <ty>> <v2>, <m x i32> <mask> ; yields <m x <ty>>
+ <result> = shufflevector <vscale x n x <ty>> <v1>, <vscale x n x <ty>> v2, <vscale x m x i32> <mask> ; yields <vscale x m x <ty>>
Overview:
"""""""""
@@ -8249,6 +8269,10 @@ undef. If any element of the mask operan
result is undef. If the shuffle mask selects an undef element from one
of the input vectors, the resulting element is undef.
+For scalable vectors, the only valid mask values at present are
+``zeroinitializer`` and ``undef``, since we cannot write all indices as
+literals for a vector with a length unknown at compile time.
+
Example:
""""""""
Modified: llvm/trunk/include/llvm/ADT/DenseMapInfo.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ADT/DenseMapInfo.h?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/include/llvm/ADT/DenseMapInfo.h (original)
+++ llvm/trunk/include/llvm/ADT/DenseMapInfo.h Wed May 29 05:22:54 2019
@@ -17,6 +17,7 @@
#include "llvm/ADT/Hashing.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/Support/PointerLikeTypeTraits.h"
+#include "llvm/Support/ScalableSize.h"
#include <cassert>
#include <cstddef>
#include <cstdint>
@@ -268,6 +269,21 @@ template <> struct DenseMapInfo<hash_cod
static bool isEqual(hash_code LHS, hash_code RHS) { return LHS == RHS; }
};
+template <> struct DenseMapInfo<ElementCount> {
+ static inline ElementCount getEmptyKey() { return {~0U, true}; }
+ static inline ElementCount getTombstoneKey() { return {~0U - 1, false}; }
+ static unsigned getHashValue(const ElementCount& EltCnt) {
+ if (EltCnt.Scalable)
+ return (EltCnt.Min * 37U) - 1U;
+
+ return EltCnt.Min * 37U;
+ }
+
+ static bool isEqual(const ElementCount& LHS, const ElementCount& RHS) {
+ return LHS == RHS;
+ }
+};
+
} // end namespace llvm
#endif // LLVM_ADT_DENSEMAPINFO_H
Modified: llvm/trunk/include/llvm/IR/DerivedTypes.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/DerivedTypes.h?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/include/llvm/IR/DerivedTypes.h (original)
+++ llvm/trunk/include/llvm/IR/DerivedTypes.h Wed May 29 05:22:54 2019
@@ -23,6 +23,7 @@
#include "llvm/IR/Type.h"
#include "llvm/Support/Casting.h"
#include "llvm/Support/Compiler.h"
+#include "llvm/Support/ScalableSize.h"
#include <cassert>
#include <cstdint>
@@ -387,6 +388,8 @@ public:
SequentialType(const SequentialType &) = delete;
SequentialType &operator=(const SequentialType &) = delete;
+ /// For scalable vectors, this will return the minimum number of elements
+ /// in the vector.
uint64_t getNumElements() const { return NumElements; }
Type *getElementType() const { return ContainedType; }
@@ -422,14 +425,37 @@ uint64_t Type::getArrayNumElements() con
/// Class to represent vector types.
class VectorType : public SequentialType {
- VectorType(Type *ElType, unsigned NumEl);
+ /// A fully specified VectorType is of the form <vscale x n x Ty>. 'n' is the
+ /// minimum number of elements of type Ty contained within the vector, and
+ /// 'scalable' indicates that the total element count is an integer multiple
+ /// of 'n', where the multiple is either guaranteed to be one, or is
+ /// statically unknown at compile time.
+ ///
+ /// If the multiple is known to be 1, then the extra term is discarded in
+ /// textual IR:
+ ///
+ /// <4 x i32> - a vector containing 4 i32s
+ /// <vscale x 4 x i32> - a vector containing an unknown integer multiple
+ /// of 4 i32s
+
+ VectorType(Type *ElType, unsigned NumEl, bool Scalable = false);
+ VectorType(Type *ElType, ElementCount EC);
+
+ // If true, the total number of elements is an unknown multiple of the
+ // minimum 'NumElements' from SequentialType. Otherwise the total number
+ // of elements is exactly equal to 'NumElements'.
+ bool Scalable;
public:
VectorType(const VectorType &) = delete;
VectorType &operator=(const VectorType &) = delete;
/// This static method is the primary way to construct an VectorType.
- static VectorType *get(Type *ElementType, unsigned NumElements);
+ static VectorType *get(Type *ElementType, ElementCount EC);
+ static VectorType *get(Type *ElementType, unsigned NumElements,
+ bool Scalable = false) {
+ return VectorType::get(ElementType, {NumElements, Scalable});
+ }
/// This static method gets a VectorType with the same number of elements as
/// the input type, and the element type is an integer type of the same width
@@ -438,7 +464,7 @@ public:
unsigned EltBits = VTy->getElementType()->getPrimitiveSizeInBits();
assert(EltBits && "Element size must be of a non-zero size");
Type *EltTy = IntegerType::get(VTy->getContext(), EltBits);
- return VectorType::get(EltTy, VTy->getNumElements());
+ return VectorType::get(EltTy, VTy->getElementCount());
}
/// This static method is like getInteger except that the element types are
@@ -446,7 +472,7 @@ public:
static VectorType *getExtendedElementVectorType(VectorType *VTy) {
unsigned EltBits = VTy->getElementType()->getPrimitiveSizeInBits();
Type *EltTy = IntegerType::get(VTy->getContext(), EltBits * 2);
- return VectorType::get(EltTy, VTy->getNumElements());
+ return VectorType::get(EltTy, VTy->getElementCount());
}
/// This static method is like getInteger except that the element types are
@@ -456,29 +482,45 @@ public:
assert((EltBits & 1) == 0 &&
"Cannot truncate vector element with odd bit-width");
Type *EltTy = IntegerType::get(VTy->getContext(), EltBits / 2);
- return VectorType::get(EltTy, VTy->getNumElements());
+ return VectorType::get(EltTy, VTy->getElementCount());
}
/// This static method returns a VectorType with half as many elements as the
/// input type and the same element type.
static VectorType *getHalfElementsVectorType(VectorType *VTy) {
- unsigned NumElts = VTy->getNumElements();
- assert ((NumElts & 1) == 0 &&
+ auto EltCnt = VTy->getElementCount();
+ assert ((EltCnt.Min & 1) == 0 &&
"Cannot halve vector with odd number of elements.");
- return VectorType::get(VTy->getElementType(), NumElts/2);
+ return VectorType::get(VTy->getElementType(), EltCnt/2);
}
/// This static method returns a VectorType with twice as many elements as the
/// input type and the same element type.
static VectorType *getDoubleElementsVectorType(VectorType *VTy) {
- unsigned NumElts = VTy->getNumElements();
- return VectorType::get(VTy->getElementType(), NumElts*2);
+ auto EltCnt = VTy->getElementCount();
+ assert((VTy->getNumElements() * 2ull) <= UINT_MAX &&
+ "Too many elements in vector");
+ return VectorType::get(VTy->getElementType(), EltCnt*2);
}
/// Return true if the specified type is valid as a element type.
static bool isValidElementType(Type *ElemTy);
- /// Return the number of bits in the Vector type.
+ /// Return an ElementCount instance to represent the (possibly scalable)
+ /// number of elements in the vector.
+ ElementCount getElementCount() const {
+ uint64_t MinimumEltCnt = getNumElements();
+ assert(MinimumEltCnt <= UINT_MAX && "Too many elements in vector");
+ return { (unsigned)MinimumEltCnt, Scalable };
+ }
+
+ /// Returns whether or not this is a scalable vector (meaning the total
+ /// element count is a multiple of the minimum).
+ bool isScalable() const {
+ return Scalable;
+ }
+
+ /// Return the minimum number of bits in the Vector type.
/// Returns zero when the vector is a vector of pointers.
unsigned getBitWidth() const {
return getNumElements() * getElementType()->getPrimitiveSizeInBits();
@@ -494,6 +536,10 @@ unsigned Type::getVectorNumElements() co
return cast<VectorType>(this)->getNumElements();
}
+bool Type::getVectorIsScalable() const {
+ return cast<VectorType>(this)->isScalable();
+}
+
/// Class to represent pointers.
class PointerType : public Type {
explicit PointerType(Type *ElType, unsigned AddrSpace);
Modified: llvm/trunk/include/llvm/IR/Type.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/Type.h?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/include/llvm/IR/Type.h (original)
+++ llvm/trunk/include/llvm/IR/Type.h Wed May 29 05:22:54 2019
@@ -366,6 +366,7 @@ public:
return ContainedTys[0];
}
+ inline bool getVectorIsScalable() const;
inline unsigned getVectorNumElements() const;
Type *getVectorElementType() const {
assert(getTypeID() == VectorTyID);
Added: llvm/trunk/include/llvm/Support/ScalableSize.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/ScalableSize.h?rev=361953&view=auto
==============================================================================
--- llvm/trunk/include/llvm/Support/ScalableSize.h (added)
+++ llvm/trunk/include/llvm/Support/ScalableSize.h Wed May 29 05:22:54 2019
@@ -0,0 +1,43 @@
+//===- ScalableSize.h - Scalable vector size info ---------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides a struct that can be used to query the size of IR types
+// which may be scalable vectors. It provides convenience operators so that
+// it can be used in much the same way as a single scalar value.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_SUPPORT_SCALABLESIZE_H
+#define LLVM_SUPPORT_SCALABLESIZE_H
+
+namespace llvm {
+
+class ElementCount {
+public:
+ unsigned Min; // Minimum number of vector elements.
+ bool Scalable; // If true, NumElements is a multiple of 'Min' determined
+ // at runtime rather than compile time.
+
+ ElementCount(unsigned Min, bool Scalable)
+ : Min(Min), Scalable(Scalable) {}
+
+ ElementCount operator*(unsigned RHS) {
+ return { Min * RHS, Scalable };
+ }
+ ElementCount operator/(unsigned RHS) {
+ return { Min / RHS, Scalable };
+ }
+
+ bool operator==(const ElementCount& RHS) const {
+ return Min == RHS.Min && Scalable == RHS.Scalable;
+ }
+};
+
+} // end namespace llvm
+
+#endif // LLVM_SUPPORT_SCALABLESIZE_H
Modified: llvm/trunk/lib/AsmParser/LLLexer.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/AsmParser/LLLexer.cpp?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/AsmParser/LLLexer.cpp (original)
+++ llvm/trunk/lib/AsmParser/LLLexer.cpp Wed May 29 05:22:54 2019
@@ -707,6 +707,7 @@ lltok::Kind LLLexer::LexIdentifier() {
KEYWORD(xchg); KEYWORD(nand); KEYWORD(max); KEYWORD(min); KEYWORD(umax);
KEYWORD(umin);
+ KEYWORD(vscale);
KEYWORD(x);
KEYWORD(blockaddress);
Modified: llvm/trunk/lib/AsmParser/LLParser.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/AsmParser/LLParser.cpp?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/AsmParser/LLParser.cpp (original)
+++ llvm/trunk/lib/AsmParser/LLParser.cpp Wed May 29 05:22:54 2019
@@ -2721,7 +2721,18 @@ bool LLParser::ParseStructBody(SmallVect
/// Type
/// ::= '[' APSINTVAL 'x' Types ']'
/// ::= '<' APSINTVAL 'x' Types '>'
+/// ::= '<' 'vscale' 'x' APSINTVAL 'x' Types '>'
bool LLParser::ParseArrayVectorType(Type *&Result, bool isVector) {
+ bool Scalable = false;
+
+ if (isVector && Lex.getKind() == lltok::kw_vscale) {
+ Lex.Lex(); // consume the 'vscale'
+ if (ParseToken(lltok::kw_x, "expected 'x' after vscale"))
+ return true;
+
+ Scalable = true;
+ }
+
if (Lex.getKind() != lltok::APSInt || Lex.getAPSIntVal().isSigned() ||
Lex.getAPSIntVal().getBitWidth() > 64)
return TokError("expected number in address space");
@@ -2748,7 +2759,7 @@ bool LLParser::ParseArrayVectorType(Type
return Error(SizeLoc, "size too large for vector");
if (!VectorType::isValidElementType(EltTy))
return Error(TypeLoc, "invalid vector element type");
- Result = VectorType::get(EltTy, unsigned(Size));
+ Result = VectorType::get(EltTy, unsigned(Size), Scalable);
} else {
if (!ArrayType::isValidElementType(EltTy))
return Error(TypeLoc, "invalid array element type");
Modified: llvm/trunk/lib/AsmParser/LLToken.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/AsmParser/LLToken.h?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/AsmParser/LLToken.h (original)
+++ llvm/trunk/lib/AsmParser/LLToken.h Wed May 29 05:22:54 2019
@@ -37,6 +37,7 @@ enum Kind {
bar, // |
colon, // :
+ kw_vscale,
kw_x,
kw_true,
kw_false,
Modified: llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp (original)
+++ llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp Wed May 29 05:22:54 2019
@@ -1758,7 +1758,8 @@ Error BitcodeReader::parseTypeTableBody(
return error("Invalid type");
ResultTy = ArrayType::get(ResultTy, Record[0]);
break;
- case bitc::TYPE_CODE_VECTOR: // VECTOR: [numelts, eltty]
+ case bitc::TYPE_CODE_VECTOR: // VECTOR: [numelts, eltty] or
+ // [numelts, eltty, scalable]
if (Record.size() < 2)
return error("Invalid record");
if (Record[0] == 0)
@@ -1766,7 +1767,8 @@ Error BitcodeReader::parseTypeTableBody(
ResultTy = getTypeByID(Record[1]);
if (!ResultTy || !StructType::isValidElementType(ResultTy))
return error("Invalid type");
- ResultTy = VectorType::get(ResultTy, Record[0]);
+ bool Scalable = Record.size() > 2 ? Record[2] : false;
+ ResultTy = VectorType::get(ResultTy, Record[0], Scalable);
break;
}
Modified: llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp (original)
+++ llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp Wed May 29 05:22:54 2019
@@ -931,10 +931,13 @@ void ModuleBitcodeWriter::writeTypeTable
}
case Type::VectorTyID: {
VectorType *VT = cast<VectorType>(T);
- // VECTOR [numelts, eltty]
+ // VECTOR [numelts, eltty] or
+ // [numelts, eltty, scalable]
Code = bitc::TYPE_CODE_VECTOR;
TypeVals.push_back(VT->getNumElements());
TypeVals.push_back(VE.getTypeID(VT->getElementType()));
+ if (VT->isScalable())
+ TypeVals.push_back(VT->isScalable());
break;
}
}
Modified: llvm/trunk/lib/IR/AsmWriter.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/AsmWriter.cpp?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/IR/AsmWriter.cpp (original)
+++ llvm/trunk/lib/IR/AsmWriter.cpp Wed May 29 05:22:54 2019
@@ -620,7 +620,10 @@ void TypePrinting::print(Type *Ty, raw_o
}
case Type::VectorTyID: {
VectorType *PTy = cast<VectorType>(Ty);
- OS << "<" << PTy->getNumElements() << " x ";
+ OS << "<";
+ if (PTy->isScalable())
+ OS << "vscale x ";
+ OS << PTy->getNumElements() << " x ";
print(PTy->getElementType(), OS);
OS << '>';
return;
Modified: llvm/trunk/lib/IR/LLVMContextImpl.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/LLVMContextImpl.h?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/IR/LLVMContextImpl.h (original)
+++ llvm/trunk/lib/IR/LLVMContextImpl.h Wed May 29 05:22:54 2019
@@ -1334,7 +1334,7 @@ public:
unsigned NamedStructTypesUniqueID = 0;
DenseMap<std::pair<Type *, uint64_t>, ArrayType*> ArrayTypes;
- DenseMap<std::pair<Type *, unsigned>, VectorType*> VectorTypes;
+ DenseMap<std::pair<Type *, ElementCount>, VectorType*> VectorTypes;
DenseMap<Type*, PointerType*> PointerTypes; // Pointers in AddrSpace = 0
DenseMap<std::pair<Type*, unsigned>, PointerType*> ASPointerTypes;
Modified: llvm/trunk/lib/IR/Type.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Type.cpp?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/IR/Type.cpp (original)
+++ llvm/trunk/lib/IR/Type.cpp Wed May 29 05:22:54 2019
@@ -599,21 +599,20 @@ bool ArrayType::isValidElementType(Type
// VectorType Implementation
//===----------------------------------------------------------------------===//
-VectorType::VectorType(Type *ElType, unsigned NumEl)
- : SequentialType(VectorTyID, ElType, NumEl) {}
+VectorType::VectorType(Type *ElType, ElementCount EC)
+ : SequentialType(VectorTyID, ElType, EC.Min), Scalable(EC.Scalable) {}
-VectorType *VectorType::get(Type *ElementType, unsigned NumElements) {
- assert(NumElements > 0 && "#Elements of a VectorType must be greater than 0");
+VectorType *VectorType::get(Type *ElementType, ElementCount EC) {
+ assert(EC.Min > 0 && "#Elements of a VectorType must be greater than 0");
assert(isValidElementType(ElementType) && "Element type of a VectorType must "
"be an integer, floating point, or "
"pointer type.");
LLVMContextImpl *pImpl = ElementType->getContext().pImpl;
VectorType *&Entry = ElementType->getContext().pImpl
- ->VectorTypes[std::make_pair(ElementType, NumElements)];
-
+ ->VectorTypes[std::make_pair(ElementType, EC)];
if (!Entry)
- Entry = new (pImpl->Alloc) VectorType(ElementType, NumElements);
+ Entry = new (pImpl->Alloc) VectorType(ElementType, EC);
return Entry;
}
Modified: llvm/trunk/lib/IR/Verifier.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Verifier.cpp?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/lib/IR/Verifier.cpp (original)
+++ llvm/trunk/lib/IR/Verifier.cpp Wed May 29 05:22:54 2019
@@ -43,6 +43,7 @@
//
//===----------------------------------------------------------------------===//
+#include "LLVMContextImpl.h"
#include "llvm/IR/Verifier.h"
#include "llvm/ADT/APFloat.h"
#include "llvm/ADT/APInt.h"
@@ -307,6 +308,7 @@ class Verifier : public InstVisitor<Veri
TBAAVerifier TBAAVerifyHelper;
void checkAtomicMemAccessSize(Type *Ty, const Instruction *I);
+ static bool containsScalableVectorValue(const Type *Ty);
public:
explicit Verifier(raw_ostream *OS, bool ShouldTreatBrokenDebugInfoAsError,
@@ -318,6 +320,33 @@ public:
bool hasBrokenDebugInfo() const { return BrokenDebugInfo; }
+ bool verifyTypes(const Module &M) {
+ LLVMContext &Ctx = M.getContext();
+ for (auto &Entry : Ctx.pImpl->ArrayTypes) {
+ ArrayType *ATy = Entry.second;
+ if (containsScalableVectorValue(ATy)) {
+ CheckFailed("Arrays cannot contain scalable vectors", ATy, &M);
+ Broken = true;
+ }
+ }
+
+ for (StructType* STy : Ctx.pImpl->AnonStructTypes)
+ if (containsScalableVectorValue(STy)) {
+ CheckFailed("Structs cannot contain scalable vectors", STy, &M);
+ Broken = true;
+ }
+
+ for (auto &Entry : Ctx.pImpl->NamedStructTypes) {
+ StructType *STy = Entry.second;
+ if (containsScalableVectorValue(STy)) {
+ CheckFailed("Structs cannot contain scalable vectors", STy, &M);
+ Broken = true;
+ }
+ }
+
+ return !Broken;
+ }
+
bool verify(const Function &F) {
assert(F.getParent() == &M &&
"An instance of this class only works with a specific module!");
@@ -387,6 +416,8 @@ public:
verifyCompileUnits();
+ verifyTypes(M);
+
verifyDeoptimizeCallingConvs();
DISubprogramAttachments.clear();
return !Broken;
@@ -613,6 +644,35 @@ void Verifier::visitGlobalValue(const Gl
});
}
+// Check for a scalable vector type, making sure to look through arrays and
+// structs. Pointers to scalable vectors don't count, since we know what the
+// size of a pointer is.
+static bool containsScalableVectorValueRecursive(const Type *Ty,
+ SmallVectorImpl<const Type*> &Visited) {
+ if (is_contained(Visited, Ty))
+ return false;
+
+ Visited.push_back(Ty);
+
+ if (auto *VTy = dyn_cast<VectorType>(Ty))
+ return VTy->isScalable();
+
+ if (auto *ATy = dyn_cast<ArrayType>(Ty))
+ return containsScalableVectorValueRecursive(ATy->getElementType(), Visited);
+
+ if (auto *STy = dyn_cast<StructType>(Ty))
+ for (Type *EltTy : STy->elements())
+ if (containsScalableVectorValueRecursive(EltTy, Visited))
+ return true;
+
+ return false;
+}
+
+bool Verifier::containsScalableVectorValue(const Type *Ty) {
+ SmallVector<const Type*, 16> VisitedList = {};
+ return containsScalableVectorValueRecursive(Ty, VisitedList);
+}
+
void Verifier::visitGlobalVariable(const GlobalVariable &GV) {
if (GV.hasInitializer()) {
Assert(GV.getInitializer()->getType() == GV.getValueType(),
@@ -691,6 +751,12 @@ void Verifier::visitGlobalVariable(const
"DIGlobalVariableExpression");
}
+ // Scalable vectors cannot be global variables, since we don't know
+ // the runtime size. Need to look inside structs/arrays to find the
+ // underlying element type as well.
+ if (containsScalableVectorValue(GV.getValueType()))
+ CheckFailed("Globals cannot contain scalable vectors", &GV);
+
if (!GV.hasInitializer()) {
visitGlobalValue(GV);
return;
Modified: llvm/trunk/test/Bitcode/compatibility.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Bitcode/compatibility.ll?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/test/Bitcode/compatibility.ll (original)
+++ llvm/trunk/test/Bitcode/compatibility.ll Wed May 29 05:22:54 2019
@@ -917,6 +917,10 @@ define void @typesystem() {
; CHECK: %t7 = alloca x86_mmx
%t8 = alloca %opaquety*
; CHECK: %t8 = alloca %opaquety*
+ %t9 = alloca <4 x i32>
+ ; CHECK: %t9 = alloca <4 x i32>
+ %t10 = alloca <vscale x 4 x i32>
+ ; CHECK: %t10 = alloca <vscale x 4 x i32>
ret void
}
Added: llvm/trunk/test/Verifier/scalable-aggregates.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Verifier/scalable-aggregates.ll?rev=361953&view=auto
==============================================================================
--- llvm/trunk/test/Verifier/scalable-aggregates.ll (added)
+++ llvm/trunk/test/Verifier/scalable-aggregates.ll Wed May 29 05:22:54 2019
@@ -0,0 +1,31 @@
+; RUN: not opt -S -verify < %s 2>&1 | FileCheck %s
+
+;; Arrays and Structs cannot contain scalable vectors, since we don't
+;; know the size at compile time and the container types need to have
+;; a known size.
+
+; CHECK-DAG: Arrays cannot contain scalable vectors
+; CHECK-DAG: [2 x { i32, <vscale x 1 x i32> }]; ModuleID = '<stdin>'
+; CHECK-DAG: Arrays cannot contain scalable vectors
+; CHECK-DAG: [4 x <vscale x 256 x i1>]; ModuleID = '<stdin>'
+; CHECK-DAG: Arrays cannot contain scalable vectors
+; CHECK-DAG: [2 x <vscale x 4 x i32>]; ModuleID = '<stdin>'
+; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG: { i64, [4 x <vscale x 256 x i1>] }; ModuleID = '<stdin>'
+; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG: { i32, <vscale x 1 x i32> }; ModuleID = '<stdin>'
+; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG: { <vscale x 16 x i8>, <vscale x 2 x double> }; ModuleID = '<stdin>'
+; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG: %sty = type { i64, <vscale x 32 x i16> }; ModuleID = '<stdin>'
+
+%sty = type { i64, <vscale x 32 x i16> }
+
+define void @scalable_aggregates() {
+ %array = alloca [2 x <vscale x 4 x i32>]
+ %struct = alloca { <vscale x 16 x i8>, <vscale x 2 x double> }
+ %named_struct = alloca %sty
+ %s_in_a = alloca [2 x { i32, <vscale x 1 x i32> } ]
+ %a_in_s = alloca { i64, [4 x <vscale x 256 x i1> ] }
+ ret void
+}
\ No newline at end of file
Added: llvm/trunk/test/Verifier/scalable-global-vars.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Verifier/scalable-global-vars.ll?rev=361953&view=auto
==============================================================================
--- llvm/trunk/test/Verifier/scalable-global-vars.ll (added)
+++ llvm/trunk/test/Verifier/scalable-global-vars.ll Wed May 29 05:22:54 2019
@@ -0,0 +1,24 @@
+; RUN: not opt -S -verify < %s 2>&1 | FileCheck %s
+
+;; Global variables cannot be scalable vectors, since we don't
+;; know the size at compile time.
+
+; CHECK: Globals cannot contain scalable vectors
+; CHECK-NEXT: <vscale x 4 x i32>* @ScalableVecGlobal
+ at ScalableVecGlobal = global <vscale x 4 x i32> zeroinitializer
+
+; CHECK: Globals cannot contain scalable vectors
+; CHECK-NEXT: [64 x <vscale x 2 x double>]* @ScalableVecGlobalArray
+ at ScalableVecGlobalArray = global [64 x <vscale x 2 x double>] zeroinitializer
+
+; CHECK: Globals cannot contain scalable vectors
+; CHECK-NEXT: { <vscale x 16 x i64>, <vscale x 16 x i1> }* @ScalableVecGlobalStruct
+ at ScalableVecGlobalStruct = global { <vscale x 16 x i64>, <vscale x 16 x i1> } zeroinitializer
+
+; CHECK: Globals cannot contain scalable vectors
+; CHECK-NEXT: { [4 x i32], [2 x { <vscale x 4 x i64>, <vscale x 32 x i8> }] }* @ScalableVecMixed
+ at ScalableVecMixed = global { [4 x i32], [2 x { <vscale x 4 x i64>, <vscale x 32 x i8> }]} zeroinitializer
+
+;; Global _pointers_ to scalable vectors are fine
+; CHECK-NOT: Globals cannot contain scalable vectors
+ at ScalableVecPtr = global <vscale x 8 x i16>* zeroinitializer
Modified: llvm/trunk/unittests/IR/CMakeLists.txt
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/IR/CMakeLists.txt?rev=361953&r1=361952&r2=361953&view=diff
==============================================================================
--- llvm/trunk/unittests/IR/CMakeLists.txt (original)
+++ llvm/trunk/unittests/IR/CMakeLists.txt Wed May 29 05:22:54 2019
@@ -37,6 +37,7 @@ add_llvm_unittest(IRTests
ValueHandleTest.cpp
ValueMapTest.cpp
ValueTest.cpp
+ VectorTypesTest.cpp
VerifierTest.cpp
WaymarkTest.cpp
)
Added: llvm/trunk/unittests/IR/VectorTypesTest.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/IR/VectorTypesTest.cpp?rev=361953&view=auto
==============================================================================
--- llvm/trunk/unittests/IR/VectorTypesTest.cpp (added)
+++ llvm/trunk/unittests/IR/VectorTypesTest.cpp Wed May 29 05:22:54 2019
@@ -0,0 +1,164 @@
+//===--- llvm/unittest/IR/VectorTypesTest.cpp - vector types unit tests ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/IR/DerivedTypes.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/Support/ScalableSize.h"
+#include "gtest/gtest.h"
+using namespace llvm;
+
+namespace {
+TEST(VectorTypesTest, FixedLength) {
+ LLVMContext Ctx;
+
+ Type *Int16Ty = Type::getInt16Ty(Ctx);
+ Type *Int32Ty = Type::getInt32Ty(Ctx);
+ Type *Int64Ty = Type::getInt64Ty(Ctx);
+ Type *Float64Ty = Type::getDoubleTy(Ctx);
+
+ VectorType *V8Int32Ty = VectorType::get(Int32Ty, 8);
+ ASSERT_FALSE(V8Int32Ty->isScalable());
+ EXPECT_EQ(V8Int32Ty->getNumElements(), 8U);
+ EXPECT_EQ(V8Int32Ty->getElementType()->getScalarSizeInBits(), 32U);
+
+ VectorType *V8Int16Ty = VectorType::get(Int16Ty, {8, false});
+ ASSERT_FALSE(V8Int16Ty->isScalable());
+ EXPECT_EQ(V8Int16Ty->getNumElements(), 8U);
+ EXPECT_EQ(V8Int16Ty->getElementType()->getScalarSizeInBits(), 16U);
+
+ ElementCount EltCnt(4, false);
+ VectorType *V4Int64Ty = VectorType::get(Int64Ty, EltCnt);
+ ASSERT_FALSE(V4Int64Ty->isScalable());
+ EXPECT_EQ(V4Int64Ty->getNumElements(), 4U);
+ EXPECT_EQ(V4Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *V2Int64Ty = VectorType::get(Int64Ty, EltCnt/2);
+ ASSERT_FALSE(V2Int64Ty->isScalable());
+ EXPECT_EQ(V2Int64Ty->getNumElements(), 2U);
+ EXPECT_EQ(V2Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *V8Int64Ty = VectorType::get(Int64Ty, EltCnt*2);
+ ASSERT_FALSE(V8Int64Ty->isScalable());
+ EXPECT_EQ(V8Int64Ty->getNumElements(), 8U);
+ EXPECT_EQ(V8Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *V4Float64Ty = VectorType::get(Float64Ty, EltCnt);
+ ASSERT_FALSE(V4Float64Ty->isScalable());
+ EXPECT_EQ(V4Float64Ty->getNumElements(), 4U);
+ EXPECT_EQ(V4Float64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *ExtTy = VectorType::getExtendedElementVectorType(V8Int16Ty);
+ EXPECT_EQ(ExtTy, V8Int32Ty);
+ ASSERT_FALSE(ExtTy->isScalable());
+ EXPECT_EQ(ExtTy->getNumElements(), 8U);
+ EXPECT_EQ(ExtTy->getElementType()->getScalarSizeInBits(), 32U);
+
+ VectorType *TruncTy = VectorType::getTruncatedElementVectorType(V8Int32Ty);
+ EXPECT_EQ(TruncTy, V8Int16Ty);
+ ASSERT_FALSE(TruncTy->isScalable());
+ EXPECT_EQ(TruncTy->getNumElements(), 8U);
+ EXPECT_EQ(TruncTy->getElementType()->getScalarSizeInBits(), 16U);
+
+ VectorType *HalvedTy = VectorType::getHalfElementsVectorType(V4Int64Ty);
+ EXPECT_EQ(HalvedTy, V2Int64Ty);
+ ASSERT_FALSE(HalvedTy->isScalable());
+ EXPECT_EQ(HalvedTy->getNumElements(), 2U);
+ EXPECT_EQ(HalvedTy->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *DoubledTy = VectorType::getDoubleElementsVectorType(V4Int64Ty);
+ EXPECT_EQ(DoubledTy, V8Int64Ty);
+ ASSERT_FALSE(DoubledTy->isScalable());
+ EXPECT_EQ(DoubledTy->getNumElements(), 8U);
+ EXPECT_EQ(DoubledTy->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *ConvTy = VectorType::getInteger(V4Float64Ty);
+ EXPECT_EQ(ConvTy, V4Int64Ty);
+ ASSERT_FALSE(ConvTy->isScalable());
+ EXPECT_EQ(ConvTy->getNumElements(), 4U);
+ EXPECT_EQ(ConvTy->getElementType()->getScalarSizeInBits(), 64U);
+
+ EltCnt = V8Int64Ty->getElementCount();
+ EXPECT_EQ(EltCnt.Min, 8U);
+ ASSERT_FALSE(EltCnt.Scalable);
+}
+
+TEST(VectorTypesTest, Scalable) {
+ LLVMContext Ctx;
+
+ Type *Int16Ty = Type::getInt16Ty(Ctx);
+ Type *Int32Ty = Type::getInt32Ty(Ctx);
+ Type *Int64Ty = Type::getInt64Ty(Ctx);
+ Type *Float64Ty = Type::getDoubleTy(Ctx);
+
+ VectorType *ScV8Int32Ty = VectorType::get(Int32Ty, 8, true);
+ ASSERT_TRUE(ScV8Int32Ty->isScalable());
+ EXPECT_EQ(ScV8Int32Ty->getNumElements(), 8U);
+ EXPECT_EQ(ScV8Int32Ty->getElementType()->getScalarSizeInBits(), 32U);
+
+ VectorType *ScV8Int16Ty = VectorType::get(Int16Ty, {8, true});
+ ASSERT_TRUE(ScV8Int16Ty->isScalable());
+ EXPECT_EQ(ScV8Int16Ty->getNumElements(), 8U);
+ EXPECT_EQ(ScV8Int16Ty->getElementType()->getScalarSizeInBits(), 16U);
+
+ ElementCount EltCnt(4, true);
+ VectorType *ScV4Int64Ty = VectorType::get(Int64Ty, EltCnt);
+ ASSERT_TRUE(ScV4Int64Ty->isScalable());
+ EXPECT_EQ(ScV4Int64Ty->getNumElements(), 4U);
+ EXPECT_EQ(ScV4Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *ScV2Int64Ty = VectorType::get(Int64Ty, EltCnt/2);
+ ASSERT_TRUE(ScV2Int64Ty->isScalable());
+ EXPECT_EQ(ScV2Int64Ty->getNumElements(), 2U);
+ EXPECT_EQ(ScV2Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *ScV8Int64Ty = VectorType::get(Int64Ty, EltCnt*2);
+ ASSERT_TRUE(ScV8Int64Ty->isScalable());
+ EXPECT_EQ(ScV8Int64Ty->getNumElements(), 8U);
+ EXPECT_EQ(ScV8Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *ScV4Float64Ty = VectorType::get(Float64Ty, EltCnt);
+ ASSERT_TRUE(ScV4Float64Ty->isScalable());
+ EXPECT_EQ(ScV4Float64Ty->getNumElements(), 4U);
+ EXPECT_EQ(ScV4Float64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *ExtTy = VectorType::getExtendedElementVectorType(ScV8Int16Ty);
+ EXPECT_EQ(ExtTy, ScV8Int32Ty);
+ ASSERT_TRUE(ExtTy->isScalable());
+ EXPECT_EQ(ExtTy->getNumElements(), 8U);
+ EXPECT_EQ(ExtTy->getElementType()->getScalarSizeInBits(), 32U);
+
+ VectorType *TruncTy = VectorType::getTruncatedElementVectorType(ScV8Int32Ty);
+ EXPECT_EQ(TruncTy, ScV8Int16Ty);
+ ASSERT_TRUE(TruncTy->isScalable());
+ EXPECT_EQ(TruncTy->getNumElements(), 8U);
+ EXPECT_EQ(TruncTy->getElementType()->getScalarSizeInBits(), 16U);
+
+ VectorType *HalvedTy = VectorType::getHalfElementsVectorType(ScV4Int64Ty);
+ EXPECT_EQ(HalvedTy, ScV2Int64Ty);
+ ASSERT_TRUE(HalvedTy->isScalable());
+ EXPECT_EQ(HalvedTy->getNumElements(), 2U);
+ EXPECT_EQ(HalvedTy->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *DoubledTy = VectorType::getDoubleElementsVectorType(ScV4Int64Ty);
+ EXPECT_EQ(DoubledTy, ScV8Int64Ty);
+ ASSERT_TRUE(DoubledTy->isScalable());
+ EXPECT_EQ(DoubledTy->getNumElements(), 8U);
+ EXPECT_EQ(DoubledTy->getElementType()->getScalarSizeInBits(), 64U);
+
+ VectorType *ConvTy = VectorType::getInteger(ScV4Float64Ty);
+ EXPECT_EQ(ConvTy, ScV4Int64Ty);
+ ASSERT_TRUE(ConvTy->isScalable());
+ EXPECT_EQ(ConvTy->getNumElements(), 4U);
+ EXPECT_EQ(ConvTy->getElementType()->getScalarSizeInBits(), 64U);
+
+ EltCnt = ScV8Int64Ty->getElementCount();
+ EXPECT_EQ(EltCnt.Min, 8U);
+ ASSERT_TRUE(EltCnt.Scalable);
+}
+
+} // end anonymous namespace
More information about the llvm-commits
mailing list