[llvm] r363658 - [SVE][IR] Scalable Vector IR Type with pr42210 fix

Graham Hunter via llvm-commits llvm-commits at lists.llvm.org
Tue Jun 18 03:11:56 PDT 2019


Author: huntergr
Date: Tue Jun 18 03:11:56 2019
New Revision: 363658

URL: http://llvm.org/viewvc/llvm-project?rev=363658&view=rev
Log:
[SVE][IR] Scalable Vector IR Type with pr42210 fix

Recommit of D32530 with a few small changes:
  - Stopped recursively walking through aggregates in
    the verifier, so that we don't impose too much
    overhead on large modules under LTO (see PR42210).
  - Changed tests to match; the errors are slightly
    different since they only report the array or
    struct that actually contains a scalable vector,
    rather than all aggregates which contain one in
    a nested member.
  - Corrected an older comment

Reviewers: thakis, rengolin, sdesmalen

Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D63321

Added:
    llvm/trunk/include/llvm/Support/ScalableSize.h
    llvm/trunk/test/Verifier/scalable-aggregates.ll
    llvm/trunk/test/Verifier/scalable-global-vars.ll
    llvm/trunk/unittests/IR/VectorTypesTest.cpp
Modified:
    llvm/trunk/docs/LangRef.rst
    llvm/trunk/include/llvm/ADT/DenseMapInfo.h
    llvm/trunk/include/llvm/IR/DerivedTypes.h
    llvm/trunk/include/llvm/IR/Type.h
    llvm/trunk/lib/AsmParser/LLLexer.cpp
    llvm/trunk/lib/AsmParser/LLParser.cpp
    llvm/trunk/lib/AsmParser/LLToken.h
    llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp
    llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp
    llvm/trunk/lib/IR/AsmWriter.cpp
    llvm/trunk/lib/IR/LLVMContextImpl.h
    llvm/trunk/lib/IR/Type.cpp
    llvm/trunk/lib/IR/Verifier.cpp
    llvm/trunk/test/Bitcode/compatibility.ll
    llvm/trunk/unittests/IR/CMakeLists.txt

Modified: llvm/trunk/docs/LangRef.rst
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/LangRef.rst?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/docs/LangRef.rst (original)
+++ llvm/trunk/docs/LangRef.rst Tue Jun 18 03:11:56 2019
@@ -675,6 +675,9 @@ an optional list of attached :ref:`metad
 Variables and aliases can have a
 :ref:`Thread Local Storage Model <tls_model>`.
 
+:ref:`Scalable vectors <t_vector>` cannot be global variables or members of
+structs or arrays because their size is unknown at compile time.
+
 Syntax::
 
       @<GlobalVarName> = [Linkage] [PreemptionSpecifier] [Visibility]
@@ -2735,30 +2738,40 @@ Vector Type
 A vector type is a simple derived type that represents a vector of
 elements. Vector types are used when multiple primitive data are
 operated in parallel using a single instruction (SIMD). A vector type
-requires a size (number of elements) and an underlying primitive data
-type. Vector types are considered :ref:`first class <t_firstclass>`.
+requires a size (number of elements), an underlying primitive data type,
+and a scalable property to represent vectors where the exact hardware
+vector length is unknown at compile time. Vector types are considered
+:ref:`first class <t_firstclass>`.
 
 :Syntax:
 
 ::
 
-      < <# elements> x <elementtype> >
+      < <# elements> x <elementtype> >          ; Fixed-length vector
+      < vscale x <# elements> x <elementtype> > ; Scalable vector
 
 The number of elements is a constant integer value larger than 0;
 elementtype may be any integer, floating-point or pointer type. Vectors
-of size zero are not allowed.
+of size zero are not allowed. For scalable vectors, the total number of
+elements is a constant multiple (called vscale) of the specified number
+of elements; vscale is a positive integer that is unknown at compile time
+and the same hardware-dependent constant for all scalable vectors at run
+time. The size of a specific scalable vector type is thus constant within
+IR, even if the exact size in bytes cannot be determined until run time.
 
 :Examples:
 
-+-------------------+--------------------------------------------------+
-| ``<4 x i32>``     | Vector of 4 32-bit integer values.               |
-+-------------------+--------------------------------------------------+
-| ``<8 x float>``   | Vector of 8 32-bit floating-point values.        |
-+-------------------+--------------------------------------------------+
-| ``<2 x i64>``     | Vector of 2 64-bit integer values.               |
-+-------------------+--------------------------------------------------+
-| ``<4 x i64*>``    | Vector of 4 pointers to 64-bit integer values.   |
-+-------------------+--------------------------------------------------+
++------------------------+----------------------------------------------------+
+| ``<4 x i32>``          | Vector of 4 32-bit integer values.                 |
++------------------------+----------------------------------------------------+
+| ``<8 x float>``        | Vector of 8 32-bit floating-point values.          |
++------------------------+----------------------------------------------------+
+| ``<2 x i64>``          | Vector of 2 64-bit integer values.                 |
++------------------------+----------------------------------------------------+
+| ``<4 x i64*>``         | Vector of 4 pointers to 64-bit integer values.     |
++------------------------+----------------------------------------------------+
+| ``<vscale x 4 x i32>`` | Vector with a multiple of 4 32-bit integer values. |
++------------------------+----------------------------------------------------+
 
 .. _t_label:
 
@@ -8154,6 +8167,7 @@ Syntax:
 ::
 
       <result> = extractelement <n x <ty>> <val>, <ty2> <idx>  ; yields <ty>
+      <result> = extractelement <vscale x n x <ty>> <val>, <ty2> <idx> ; yields <ty>
 
 Overview:
 """""""""
@@ -8174,7 +8188,9 @@ Semantics:
 
 The result is a scalar of the same type as the element type of ``val``.
 Its value is the value at position ``idx`` of ``val``. If ``idx``
-exceeds the length of ``val``, the result is a
+exceeds the length of ``val`` for a fixed-length vector, the result is a
+:ref:`poison value <poisonvalues>`. For a scalable vector, if the value
+of ``idx`` exceeds the runtime length of the vector, the result is a
 :ref:`poison value <poisonvalues>`.
 
 Example:
@@ -8195,6 +8211,7 @@ Syntax:
 ::
 
       <result> = insertelement <n x <ty>> <val>, <ty> <elt>, <ty2> <idx>    ; yields <n x <ty>>
+      <result> = insertelement <vscale x n x <ty>> <val>, <ty> <elt>, <ty2> <idx> ; yields <vscale x n x <ty>>
 
 Overview:
 """""""""
@@ -8216,7 +8233,9 @@ Semantics:
 
 The result is a vector of the same type as ``val``. Its element values
 are those of ``val`` except at position ``idx``, where it gets the value
-``elt``. If ``idx`` exceeds the length of ``val``, the result
+``elt``. If ``idx`` exceeds the length of ``val`` for a fixed-length vector,
+the result is a :ref:`poison value <poisonvalues>`. For a scalable vector,
+if the value of ``idx`` exceeds the runtime length of the vector, the result
 is a :ref:`poison value <poisonvalues>`.
 
 Example:
@@ -8237,6 +8256,7 @@ Syntax:
 ::
 
       <result> = shufflevector <n x <ty>> <v1>, <n x <ty>> <v2>, <m x i32> <mask>    ; yields <m x <ty>>
+      <result> = shufflevector <vscale x n x <ty>> <v1>, <vscale x n x <ty>> v2, <vscale x m x i32> <mask>  ; yields <vscale x m x <ty>>
 
 Overview:
 """""""""
@@ -8268,6 +8288,10 @@ undef. If any element of the mask operan
 result is undef. If the shuffle mask selects an undef element from one
 of the input vectors, the resulting element is undef.
 
+For scalable vectors, the only valid mask values at present are
+``zeroinitializer`` and ``undef``, since we cannot write all indices as
+literals for a vector with a length unknown at compile time.
+
 Example:
 """"""""
 

Modified: llvm/trunk/include/llvm/ADT/DenseMapInfo.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ADT/DenseMapInfo.h?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/include/llvm/ADT/DenseMapInfo.h (original)
+++ llvm/trunk/include/llvm/ADT/DenseMapInfo.h Tue Jun 18 03:11:56 2019
@@ -17,6 +17,7 @@
 #include "llvm/ADT/Hashing.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/PointerLikeTypeTraits.h"
+#include "llvm/Support/ScalableSize.h"
 #include <cassert>
 #include <cstddef>
 #include <cstdint>
@@ -268,6 +269,21 @@ template <> struct DenseMapInfo<hash_cod
   static bool isEqual(hash_code LHS, hash_code RHS) { return LHS == RHS; }
 };
 
+template <> struct DenseMapInfo<ElementCount> {
+  static inline ElementCount getEmptyKey() { return {~0U, true}; }
+  static inline ElementCount getTombstoneKey() { return {~0U - 1, false}; }
+  static unsigned getHashValue(const ElementCount& EltCnt) {
+    if (EltCnt.Scalable)
+      return (EltCnt.Min * 37U) - 1U;
+
+    return EltCnt.Min * 37U;
+  }
+
+  static bool isEqual(const ElementCount& LHS, const ElementCount& RHS) {
+    return LHS == RHS;
+  }
+};
+
 } // end namespace llvm
 
 #endif // LLVM_ADT_DENSEMAPINFO_H

Modified: llvm/trunk/include/llvm/IR/DerivedTypes.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/DerivedTypes.h?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/include/llvm/IR/DerivedTypes.h (original)
+++ llvm/trunk/include/llvm/IR/DerivedTypes.h Tue Jun 18 03:11:56 2019
@@ -23,6 +23,7 @@
 #include "llvm/IR/Type.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/Compiler.h"
+#include "llvm/Support/ScalableSize.h"
 #include <cassert>
 #include <cstdint>
 
@@ -387,6 +388,8 @@ public:
   SequentialType(const SequentialType &) = delete;
   SequentialType &operator=(const SequentialType &) = delete;
 
+  /// For scalable vectors, this will return the minimum number of elements
+  /// in the vector.
   uint64_t getNumElements() const { return NumElements; }
   Type *getElementType() const { return ContainedType; }
 
@@ -422,14 +425,37 @@ uint64_t Type::getArrayNumElements() con
 
 /// Class to represent vector types.
 class VectorType : public SequentialType {
-  VectorType(Type *ElType, unsigned NumEl);
+  /// A fully specified VectorType is of the form <vscale x n x Ty>. 'n' is the
+  /// minimum number of elements of type Ty contained within the vector, and
+  /// 'vscale x' indicates that the total element count is an integer multiple
+  /// of 'n', where the multiple is either guaranteed to be one, or is
+  /// statically unknown at compile time.
+  ///
+  /// If the multiple is known to be 1, then the extra term is discarded in
+  /// textual IR:
+  ///
+  /// <4 x i32>          - a vector containing 4 i32s
+  /// <vscale x 4 x i32> - a vector containing an unknown integer multiple
+  ///                      of 4 i32s
+
+  VectorType(Type *ElType, unsigned NumEl, bool Scalable = false);
+  VectorType(Type *ElType, ElementCount EC);
+
+  // If true, the total number of elements is an unknown multiple of the
+  // minimum 'NumElements' from SequentialType. Otherwise the total number
+  // of elements is exactly equal to 'NumElements'.
+  bool Scalable;
 
 public:
   VectorType(const VectorType &) = delete;
   VectorType &operator=(const VectorType &) = delete;
 
   /// This static method is the primary way to construct an VectorType.
-  static VectorType *get(Type *ElementType, unsigned NumElements);
+  static VectorType *get(Type *ElementType, ElementCount EC);
+  static VectorType *get(Type *ElementType, unsigned NumElements,
+                         bool Scalable = false) {
+    return VectorType::get(ElementType, {NumElements, Scalable});
+  }
 
   /// This static method gets a VectorType with the same number of elements as
   /// the input type, and the element type is an integer type of the same width
@@ -438,7 +464,7 @@ public:
     unsigned EltBits = VTy->getElementType()->getPrimitiveSizeInBits();
     assert(EltBits && "Element size must be of a non-zero size");
     Type *EltTy = IntegerType::get(VTy->getContext(), EltBits);
-    return VectorType::get(EltTy, VTy->getNumElements());
+    return VectorType::get(EltTy, VTy->getElementCount());
   }
 
   /// This static method is like getInteger except that the element types are
@@ -446,7 +472,7 @@ public:
   static VectorType *getExtendedElementVectorType(VectorType *VTy) {
     unsigned EltBits = VTy->getElementType()->getPrimitiveSizeInBits();
     Type *EltTy = IntegerType::get(VTy->getContext(), EltBits * 2);
-    return VectorType::get(EltTy, VTy->getNumElements());
+    return VectorType::get(EltTy, VTy->getElementCount());
   }
 
   /// This static method is like getInteger except that the element types are
@@ -456,29 +482,45 @@ public:
     assert((EltBits & 1) == 0 &&
            "Cannot truncate vector element with odd bit-width");
     Type *EltTy = IntegerType::get(VTy->getContext(), EltBits / 2);
-    return VectorType::get(EltTy, VTy->getNumElements());
+    return VectorType::get(EltTy, VTy->getElementCount());
   }
 
   /// This static method returns a VectorType with half as many elements as the
   /// input type and the same element type.
   static VectorType *getHalfElementsVectorType(VectorType *VTy) {
-    unsigned NumElts = VTy->getNumElements();
-    assert ((NumElts & 1) == 0 &&
+    auto EltCnt = VTy->getElementCount();
+    assert ((EltCnt.Min & 1) == 0 &&
             "Cannot halve vector with odd number of elements.");
-    return VectorType::get(VTy->getElementType(), NumElts/2);
+    return VectorType::get(VTy->getElementType(), EltCnt/2);
   }
 
   /// This static method returns a VectorType with twice as many elements as the
   /// input type and the same element type.
   static VectorType *getDoubleElementsVectorType(VectorType *VTy) {
-    unsigned NumElts = VTy->getNumElements();
-    return VectorType::get(VTy->getElementType(), NumElts*2);
+    auto EltCnt = VTy->getElementCount();
+    assert((VTy->getNumElements() * 2ull) <= UINT_MAX &&
+           "Too many elements in vector");
+    return VectorType::get(VTy->getElementType(), EltCnt*2);
   }
 
   /// Return true if the specified type is valid as a element type.
   static bool isValidElementType(Type *ElemTy);
 
-  /// Return the number of bits in the Vector type.
+  /// Return an ElementCount instance to represent the (possibly scalable)
+  /// number of elements in the vector.
+  ElementCount getElementCount() const {
+    uint64_t MinimumEltCnt = getNumElements();
+    assert(MinimumEltCnt <= UINT_MAX && "Too many elements in vector");
+    return { (unsigned)MinimumEltCnt, Scalable };
+  }
+
+  /// Returns whether or not this is a scalable vector (meaning the total
+  /// element count is a multiple of the minimum).
+  bool isScalable() const {
+    return Scalable;
+  }
+
+  /// Return the minimum number of bits in the Vector type.
   /// Returns zero when the vector is a vector of pointers.
   unsigned getBitWidth() const {
     return getNumElements() * getElementType()->getPrimitiveSizeInBits();
@@ -494,6 +536,10 @@ unsigned Type::getVectorNumElements() co
   return cast<VectorType>(this)->getNumElements();
 }
 
+bool Type::getVectorIsScalable() const {
+  return cast<VectorType>(this)->isScalable();
+}
+
 /// Class to represent pointers.
 class PointerType : public Type {
   explicit PointerType(Type *ElType, unsigned AddrSpace);

Modified: llvm/trunk/include/llvm/IR/Type.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/Type.h?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/include/llvm/IR/Type.h (original)
+++ llvm/trunk/include/llvm/IR/Type.h Tue Jun 18 03:11:56 2019
@@ -366,6 +366,7 @@ public:
     return ContainedTys[0];
   }
 
+  inline bool getVectorIsScalable() const;
   inline unsigned getVectorNumElements() const;
   Type *getVectorElementType() const {
     assert(getTypeID() == VectorTyID);

Added: llvm/trunk/include/llvm/Support/ScalableSize.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/ScalableSize.h?rev=363658&view=auto
==============================================================================
--- llvm/trunk/include/llvm/Support/ScalableSize.h (added)
+++ llvm/trunk/include/llvm/Support/ScalableSize.h Tue Jun 18 03:11:56 2019
@@ -0,0 +1,43 @@
+//===- ScalableSize.h - Scalable vector size info ---------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides a struct that can be used to query the size of IR types
+// which may be scalable vectors. It provides convenience operators so that
+// it can be used in much the same way as a single scalar value.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_SUPPORT_SCALABLESIZE_H
+#define LLVM_SUPPORT_SCALABLESIZE_H
+
+namespace llvm {
+
+class ElementCount {
+public:
+  unsigned Min;  // Minimum number of vector elements.
+  bool Scalable; // If true, NumElements is a multiple of 'Min' determined
+                 // at runtime rather than compile time.
+
+  ElementCount(unsigned Min, bool Scalable)
+  : Min(Min), Scalable(Scalable) {}
+
+  ElementCount operator*(unsigned RHS) {
+    return { Min * RHS, Scalable };
+  }
+  ElementCount operator/(unsigned RHS) {
+    return { Min / RHS, Scalable };
+  }
+
+  bool operator==(const ElementCount& RHS) const {
+    return Min == RHS.Min && Scalable == RHS.Scalable;
+  }
+};
+
+} // end namespace llvm
+
+#endif // LLVM_SUPPORT_SCALABLESIZE_H

Modified: llvm/trunk/lib/AsmParser/LLLexer.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/AsmParser/LLLexer.cpp?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/AsmParser/LLLexer.cpp (original)
+++ llvm/trunk/lib/AsmParser/LLLexer.cpp Tue Jun 18 03:11:56 2019
@@ -707,6 +707,7 @@ lltok::Kind LLLexer::LexIdentifier() {
   KEYWORD(xchg); KEYWORD(nand); KEYWORD(max); KEYWORD(min); KEYWORD(umax);
   KEYWORD(umin);
 
+  KEYWORD(vscale);
   KEYWORD(x);
   KEYWORD(blockaddress);
 

Modified: llvm/trunk/lib/AsmParser/LLParser.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/AsmParser/LLParser.cpp?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/AsmParser/LLParser.cpp (original)
+++ llvm/trunk/lib/AsmParser/LLParser.cpp Tue Jun 18 03:11:56 2019
@@ -2743,7 +2743,18 @@ bool LLParser::ParseStructBody(SmallVect
 ///   Type
 ///     ::= '[' APSINTVAL 'x' Types ']'
 ///     ::= '<' APSINTVAL 'x' Types '>'
+///     ::= '<' 'vscale' 'x' APSINTVAL 'x' Types '>'
 bool LLParser::ParseArrayVectorType(Type *&Result, bool isVector) {
+  bool Scalable = false;
+
+  if (isVector && Lex.getKind() == lltok::kw_vscale) {
+    Lex.Lex(); // consume the 'vscale'
+    if (ParseToken(lltok::kw_x, "expected 'x' after vscale"))
+      return true;
+
+    Scalable = true;
+  }
+
   if (Lex.getKind() != lltok::APSInt || Lex.getAPSIntVal().isSigned() ||
       Lex.getAPSIntVal().getBitWidth() > 64)
     return TokError("expected number in address space");
@@ -2770,7 +2781,7 @@ bool LLParser::ParseArrayVectorType(Type
       return Error(SizeLoc, "size too large for vector");
     if (!VectorType::isValidElementType(EltTy))
       return Error(TypeLoc, "invalid vector element type");
-    Result = VectorType::get(EltTy, unsigned(Size));
+    Result = VectorType::get(EltTy, unsigned(Size), Scalable);
   } else {
     if (!ArrayType::isValidElementType(EltTy))
       return Error(TypeLoc, "invalid array element type");

Modified: llvm/trunk/lib/AsmParser/LLToken.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/AsmParser/LLToken.h?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/AsmParser/LLToken.h (original)
+++ llvm/trunk/lib/AsmParser/LLToken.h Tue Jun 18 03:11:56 2019
@@ -37,6 +37,7 @@ enum Kind {
   bar,     // |
   colon,   // :
 
+  kw_vscale,
   kw_x,
   kw_true,
   kw_false,

Modified: llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp (original)
+++ llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp Tue Jun 18 03:11:56 2019
@@ -1775,7 +1775,8 @@ Error BitcodeReader::parseTypeTableBody(
         return error("Invalid type");
       ResultTy = ArrayType::get(ResultTy, Record[0]);
       break;
-    case bitc::TYPE_CODE_VECTOR:    // VECTOR: [numelts, eltty]
+    case bitc::TYPE_CODE_VECTOR:    // VECTOR: [numelts, eltty] or
+                                    //         [numelts, eltty, scalable]
       if (Record.size() < 2)
         return error("Invalid record");
       if (Record[0] == 0)
@@ -1783,7 +1784,8 @@ Error BitcodeReader::parseTypeTableBody(
       ResultTy = getTypeByID(Record[1]);
       if (!ResultTy || !StructType::isValidElementType(ResultTy))
         return error("Invalid type");
-      ResultTy = VectorType::get(ResultTy, Record[0]);
+      bool Scalable = Record.size() > 2 ? Record[2] : false;
+      ResultTy = VectorType::get(ResultTy, Record[0], Scalable);
       break;
     }
 

Modified: llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp (original)
+++ llvm/trunk/lib/Bitcode/Writer/BitcodeWriter.cpp Tue Jun 18 03:11:56 2019
@@ -938,10 +938,13 @@ void ModuleBitcodeWriter::writeTypeTable
     }
     case Type::VectorTyID: {
       VectorType *VT = cast<VectorType>(T);
-      // VECTOR [numelts, eltty]
+      // VECTOR [numelts, eltty] or
+      //        [numelts, eltty, scalable]
       Code = bitc::TYPE_CODE_VECTOR;
       TypeVals.push_back(VT->getNumElements());
       TypeVals.push_back(VE.getTypeID(VT->getElementType()));
+      if (VT->isScalable())
+        TypeVals.push_back(VT->isScalable());
       break;
     }
     }

Modified: llvm/trunk/lib/IR/AsmWriter.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/AsmWriter.cpp?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/IR/AsmWriter.cpp (original)
+++ llvm/trunk/lib/IR/AsmWriter.cpp Tue Jun 18 03:11:56 2019
@@ -620,7 +620,10 @@ void TypePrinting::print(Type *Ty, raw_o
   }
   case Type::VectorTyID: {
     VectorType *PTy = cast<VectorType>(Ty);
-    OS << "<" << PTy->getNumElements() << " x ";
+    OS << "<";
+    if (PTy->isScalable())
+      OS << "vscale x ";
+    OS << PTy->getNumElements() << " x ";
     print(PTy->getElementType(), OS);
     OS << '>';
     return;

Modified: llvm/trunk/lib/IR/LLVMContextImpl.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/LLVMContextImpl.h?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/IR/LLVMContextImpl.h (original)
+++ llvm/trunk/lib/IR/LLVMContextImpl.h Tue Jun 18 03:11:56 2019
@@ -1334,7 +1334,7 @@ public:
   unsigned NamedStructTypesUniqueID = 0;
 
   DenseMap<std::pair<Type *, uint64_t>, ArrayType*> ArrayTypes;
-  DenseMap<std::pair<Type *, unsigned>, VectorType*> VectorTypes;
+  DenseMap<std::pair<Type *, ElementCount>, VectorType*> VectorTypes;
   DenseMap<Type*, PointerType*> PointerTypes;  // Pointers in AddrSpace = 0
   DenseMap<std::pair<Type*, unsigned>, PointerType*> ASPointerTypes;
 

Modified: llvm/trunk/lib/IR/Type.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Type.cpp?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/IR/Type.cpp (original)
+++ llvm/trunk/lib/IR/Type.cpp Tue Jun 18 03:11:56 2019
@@ -599,21 +599,20 @@ bool ArrayType::isValidElementType(Type
 //                          VectorType Implementation
 //===----------------------------------------------------------------------===//
 
-VectorType::VectorType(Type *ElType, unsigned NumEl)
-  : SequentialType(VectorTyID, ElType, NumEl) {}
+VectorType::VectorType(Type *ElType, ElementCount EC)
+  : SequentialType(VectorTyID, ElType, EC.Min), Scalable(EC.Scalable) {}
 
-VectorType *VectorType::get(Type *ElementType, unsigned NumElements) {
-  assert(NumElements > 0 && "#Elements of a VectorType must be greater than 0");
+VectorType *VectorType::get(Type *ElementType, ElementCount EC) {
+  assert(EC.Min > 0 && "#Elements of a VectorType must be greater than 0");
   assert(isValidElementType(ElementType) && "Element type of a VectorType must "
                                             "be an integer, floating point, or "
                                             "pointer type.");
 
   LLVMContextImpl *pImpl = ElementType->getContext().pImpl;
   VectorType *&Entry = ElementType->getContext().pImpl
-    ->VectorTypes[std::make_pair(ElementType, NumElements)];
-
+                                 ->VectorTypes[std::make_pair(ElementType, EC)];
   if (!Entry)
-    Entry = new (pImpl->Alloc) VectorType(ElementType, NumElements);
+    Entry = new (pImpl->Alloc) VectorType(ElementType, EC);
   return Entry;
 }
 

Modified: llvm/trunk/lib/IR/Verifier.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Verifier.cpp?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/lib/IR/Verifier.cpp (original)
+++ llvm/trunk/lib/IR/Verifier.cpp Tue Jun 18 03:11:56 2019
@@ -43,6 +43,7 @@
 //
 //===----------------------------------------------------------------------===//
 
+#include "LLVMContextImpl.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/ADT/APFloat.h"
 #include "llvm/ADT/APInt.h"
@@ -318,6 +319,31 @@ public:
 
   bool hasBrokenDebugInfo() const { return BrokenDebugInfo; }
 
+  void verifyTypes() {
+    LLVMContext &Ctx = M.getContext();
+    for (auto &Entry : Ctx.pImpl->ArrayTypes) {
+      Type *EltTy = Entry.second->getElementType();
+      if (auto *VTy = dyn_cast<VectorType>(EltTy))
+        if (VTy->isScalable())
+          CheckFailed("Arrays cannot contain scalable vectors",
+                      Entry.second, &M);
+    }
+
+    for (StructType* STy : Ctx.pImpl->AnonStructTypes)
+      for (Type *EltTy : STy->elements())
+        if (auto *VTy = dyn_cast<VectorType>(EltTy))
+          if (VTy->isScalable())
+            CheckFailed("Structs cannot contain scalable vectors", STy, &M);
+
+    for (auto &Entry : Ctx.pImpl->NamedStructTypes) {
+      StructType *STy = Entry.second;
+      for (Type *EltTy : STy->elements())
+        if (auto *VTy = dyn_cast<VectorType>(EltTy))
+          if (VTy->isScalable())
+            CheckFailed("Structs cannot contain scalable vectors", STy, &M);
+    }
+  }
+
   bool verify(const Function &F) {
     assert(F.getParent() == &M &&
            "An instance of this class only works with a specific module!");
@@ -387,6 +413,8 @@ public:
 
     verifyCompileUnits();
 
+    verifyTypes();
+
     verifyDeoptimizeCallingConvs();
     DISubprogramAttachments.clear();
     return !Broken;
@@ -691,6 +719,13 @@ void Verifier::visitGlobalVariable(const
                       "DIGlobalVariableExpression");
   }
 
+  // Scalable vectors cannot be global variables, since we don't know
+  // the runtime size. If the global is a struct or an array containing
+  // scalable vectors, that will be caught be verifyTypes instead.
+  if (auto *VTy = dyn_cast<VectorType>(GV.getValueType()))
+    if (VTy->isScalable())
+      CheckFailed("Globals cannot contain scalable vectors", &GV);
+
   if (!GV.hasInitializer()) {
     visitGlobalValue(GV);
     return;

Modified: llvm/trunk/test/Bitcode/compatibility.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Bitcode/compatibility.ll?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/test/Bitcode/compatibility.ll (original)
+++ llvm/trunk/test/Bitcode/compatibility.ll Tue Jun 18 03:11:56 2019
@@ -917,6 +917,10 @@ define void @typesystem() {
   ; CHECK: %t7 = alloca x86_mmx
   %t8 = alloca %opaquety*
   ; CHECK: %t8 = alloca %opaquety*
+  %t9 = alloca <4 x i32>
+  ; CHECK: %t9 = alloca <4 x i32>
+  %t10 = alloca <vscale x 4 x i32>
+  ; CHECK: %t10 = alloca <vscale x 4 x i32>
 
   ret void
 }

Added: llvm/trunk/test/Verifier/scalable-aggregates.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Verifier/scalable-aggregates.ll?rev=363658&view=auto
==============================================================================
--- llvm/trunk/test/Verifier/scalable-aggregates.ll (added)
+++ llvm/trunk/test/Verifier/scalable-aggregates.ll Tue Jun 18 03:11:56 2019
@@ -0,0 +1,27 @@
+; RUN: not opt -S -verify < %s 2>&1 | FileCheck %s
+
+;; Arrays and Structs cannot contain scalable vectors, since we don't
+;; know the size at compile time and the container types need to have
+;; a known size.
+
+; CHECK-DAG: Arrays cannot contain scalable vectors
+; CHECK-DAG:  [4 x <vscale x 256 x i1>]; ModuleID = '<stdin>'
+; CHECK-DAG: Arrays cannot contain scalable vectors
+; CHECK-DAG:  [2 x <vscale x 4 x i32>]; ModuleID = '<stdin>'
+; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG:  { i32, <vscale x 1 x i32> }; ModuleID = '<stdin>'
+;; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG: { <vscale x 16 x i8>, <vscale x 2 x double> }; ModuleID = '<stdin>'
+; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG:  %sty = type { i64, <vscale x 32 x i16> }; ModuleID = '<stdin>'
+
+%sty = type { i64, <vscale x 32 x i16> }
+
+define void @scalable_aggregates() {
+  %array = alloca [2 x <vscale x 4 x i32>]
+  %struct = alloca { <vscale x 16 x i8>, <vscale x 2 x double> }
+  %named_struct = alloca %sty
+  %s_in_a = alloca [2 x { i32, <vscale x 1 x i32> } ]
+  %a_in_s = alloca { i64, [4 x <vscale x 256 x i1> ] }
+  ret void
+}
\ No newline at end of file

Added: llvm/trunk/test/Verifier/scalable-global-vars.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Verifier/scalable-global-vars.ll?rev=363658&view=auto
==============================================================================
--- llvm/trunk/test/Verifier/scalable-global-vars.ll (added)
+++ llvm/trunk/test/Verifier/scalable-global-vars.ll Tue Jun 18 03:11:56 2019
@@ -0,0 +1,26 @@
+; RUN: not opt -S -verify < %s 2>&1 | FileCheck %s
+
+;; Global variables cannot be scalable vectors, since we don't
+;; know the size at compile time.
+
+; CHECK: Globals cannot contain scalable vectors
+; CHECK-NEXT: <vscale x 4 x i32>* @ScalableVecGlobal
+ at ScalableVecGlobal = global <vscale x 4 x i32> zeroinitializer
+
+;; Global _pointers_ to scalable vectors are fine
+; CHECK-NOT: Globals cannot contain scalable vectors
+ at ScalableVecPtr = global <vscale x 8 x i16>* zeroinitializer
+
+;; The following errors don't explicitly mention global variables, but
+;; do still guarantee that the error will be caught.
+; CHECK-DAG: Arrays cannot contain scalable vectors
+; CHECK-DAG:  [64 x <vscale x 2 x double>]; ModuleID = '<stdin>'
+ at ScalableVecGlobalArray = global [64 x <vscale x 2 x double>] zeroinitializer
+
+; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG:  { <vscale x 16 x i64>, <vscale x 16 x i1> }; ModuleID = '<stdin>'
+ at ScalableVecGlobalStruct = global { <vscale x 16 x i64>, <vscale x 16 x i1> } zeroinitializer
+
+; CHECK-DAG: Structs cannot contain scalable vectors
+; CHECK-DAG  { <vscale x 4 x i64>, <vscale x 32 x i8> }; ModuleID = '<stdin>'
+ at ScalableVecMixed = global { [4 x i32], [2 x { <vscale x 4 x i64>,  <vscale x 32 x i8> }]} zeroinitializer

Modified: llvm/trunk/unittests/IR/CMakeLists.txt
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/IR/CMakeLists.txt?rev=363658&r1=363657&r2=363658&view=diff
==============================================================================
--- llvm/trunk/unittests/IR/CMakeLists.txt (original)
+++ llvm/trunk/unittests/IR/CMakeLists.txt Tue Jun 18 03:11:56 2019
@@ -37,6 +37,7 @@ add_llvm_unittest(IRTests
   ValueHandleTest.cpp
   ValueMapTest.cpp
   ValueTest.cpp
+  VectorTypesTest.cpp
   VerifierTest.cpp
   WaymarkTest.cpp
   )

Added: llvm/trunk/unittests/IR/VectorTypesTest.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/IR/VectorTypesTest.cpp?rev=363658&view=auto
==============================================================================
--- llvm/trunk/unittests/IR/VectorTypesTest.cpp (added)
+++ llvm/trunk/unittests/IR/VectorTypesTest.cpp Tue Jun 18 03:11:56 2019
@@ -0,0 +1,164 @@
+//===--- llvm/unittest/IR/VectorTypesTest.cpp - vector types unit tests ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/IR/DerivedTypes.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/Support/ScalableSize.h"
+#include "gtest/gtest.h"
+using namespace llvm;
+
+namespace {
+TEST(VectorTypesTest, FixedLength) {
+  LLVMContext Ctx;
+
+  Type *Int16Ty = Type::getInt16Ty(Ctx);
+  Type *Int32Ty = Type::getInt32Ty(Ctx);
+  Type *Int64Ty = Type::getInt64Ty(Ctx);
+  Type *Float64Ty = Type::getDoubleTy(Ctx);
+
+  VectorType *V8Int32Ty = VectorType::get(Int32Ty, 8);
+  ASSERT_FALSE(V8Int32Ty->isScalable());
+  EXPECT_EQ(V8Int32Ty->getNumElements(), 8U);
+  EXPECT_EQ(V8Int32Ty->getElementType()->getScalarSizeInBits(), 32U);
+
+  VectorType *V8Int16Ty = VectorType::get(Int16Ty, {8, false});
+  ASSERT_FALSE(V8Int16Ty->isScalable());
+  EXPECT_EQ(V8Int16Ty->getNumElements(), 8U);
+  EXPECT_EQ(V8Int16Ty->getElementType()->getScalarSizeInBits(), 16U);
+
+  ElementCount EltCnt(4, false);
+  VectorType *V4Int64Ty = VectorType::get(Int64Ty, EltCnt);
+  ASSERT_FALSE(V4Int64Ty->isScalable());
+  EXPECT_EQ(V4Int64Ty->getNumElements(), 4U);
+  EXPECT_EQ(V4Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *V2Int64Ty = VectorType::get(Int64Ty, EltCnt/2);
+  ASSERT_FALSE(V2Int64Ty->isScalable());
+  EXPECT_EQ(V2Int64Ty->getNumElements(), 2U);
+  EXPECT_EQ(V2Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *V8Int64Ty = VectorType::get(Int64Ty, EltCnt*2);
+  ASSERT_FALSE(V8Int64Ty->isScalable());
+  EXPECT_EQ(V8Int64Ty->getNumElements(), 8U);
+  EXPECT_EQ(V8Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *V4Float64Ty = VectorType::get(Float64Ty, EltCnt);
+  ASSERT_FALSE(V4Float64Ty->isScalable());
+  EXPECT_EQ(V4Float64Ty->getNumElements(), 4U);
+  EXPECT_EQ(V4Float64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *ExtTy = VectorType::getExtendedElementVectorType(V8Int16Ty);
+  EXPECT_EQ(ExtTy, V8Int32Ty);
+  ASSERT_FALSE(ExtTy->isScalable());
+  EXPECT_EQ(ExtTy->getNumElements(), 8U);
+  EXPECT_EQ(ExtTy->getElementType()->getScalarSizeInBits(), 32U);
+
+  VectorType *TruncTy = VectorType::getTruncatedElementVectorType(V8Int32Ty);
+  EXPECT_EQ(TruncTy, V8Int16Ty);
+  ASSERT_FALSE(TruncTy->isScalable());
+  EXPECT_EQ(TruncTy->getNumElements(), 8U);
+  EXPECT_EQ(TruncTy->getElementType()->getScalarSizeInBits(), 16U);
+
+  VectorType *HalvedTy = VectorType::getHalfElementsVectorType(V4Int64Ty);
+  EXPECT_EQ(HalvedTy, V2Int64Ty);
+  ASSERT_FALSE(HalvedTy->isScalable());
+  EXPECT_EQ(HalvedTy->getNumElements(), 2U);
+  EXPECT_EQ(HalvedTy->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *DoubledTy = VectorType::getDoubleElementsVectorType(V4Int64Ty);
+  EXPECT_EQ(DoubledTy, V8Int64Ty);
+  ASSERT_FALSE(DoubledTy->isScalable());
+  EXPECT_EQ(DoubledTy->getNumElements(), 8U);
+  EXPECT_EQ(DoubledTy->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *ConvTy = VectorType::getInteger(V4Float64Ty);
+  EXPECT_EQ(ConvTy, V4Int64Ty);
+  ASSERT_FALSE(ConvTy->isScalable());
+  EXPECT_EQ(ConvTy->getNumElements(), 4U);
+  EXPECT_EQ(ConvTy->getElementType()->getScalarSizeInBits(), 64U);
+
+  EltCnt = V8Int64Ty->getElementCount();
+  EXPECT_EQ(EltCnt.Min, 8U);
+  ASSERT_FALSE(EltCnt.Scalable);
+}
+
+TEST(VectorTypesTest, Scalable) {
+  LLVMContext Ctx;
+
+  Type *Int16Ty = Type::getInt16Ty(Ctx);
+  Type *Int32Ty = Type::getInt32Ty(Ctx);
+  Type *Int64Ty = Type::getInt64Ty(Ctx);
+  Type *Float64Ty = Type::getDoubleTy(Ctx);
+
+  VectorType *ScV8Int32Ty = VectorType::get(Int32Ty, 8, true);
+  ASSERT_TRUE(ScV8Int32Ty->isScalable());
+  EXPECT_EQ(ScV8Int32Ty->getNumElements(), 8U);
+  EXPECT_EQ(ScV8Int32Ty->getElementType()->getScalarSizeInBits(), 32U);
+
+  VectorType *ScV8Int16Ty = VectorType::get(Int16Ty, {8, true});
+  ASSERT_TRUE(ScV8Int16Ty->isScalable());
+  EXPECT_EQ(ScV8Int16Ty->getNumElements(), 8U);
+  EXPECT_EQ(ScV8Int16Ty->getElementType()->getScalarSizeInBits(), 16U);
+
+  ElementCount EltCnt(4, true);
+  VectorType *ScV4Int64Ty = VectorType::get(Int64Ty, EltCnt);
+  ASSERT_TRUE(ScV4Int64Ty->isScalable());
+  EXPECT_EQ(ScV4Int64Ty->getNumElements(), 4U);
+  EXPECT_EQ(ScV4Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *ScV2Int64Ty = VectorType::get(Int64Ty, EltCnt/2);
+  ASSERT_TRUE(ScV2Int64Ty->isScalable());
+  EXPECT_EQ(ScV2Int64Ty->getNumElements(), 2U);
+  EXPECT_EQ(ScV2Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *ScV8Int64Ty = VectorType::get(Int64Ty, EltCnt*2);
+  ASSERT_TRUE(ScV8Int64Ty->isScalable());
+  EXPECT_EQ(ScV8Int64Ty->getNumElements(), 8U);
+  EXPECT_EQ(ScV8Int64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *ScV4Float64Ty = VectorType::get(Float64Ty, EltCnt);
+  ASSERT_TRUE(ScV4Float64Ty->isScalable());
+  EXPECT_EQ(ScV4Float64Ty->getNumElements(), 4U);
+  EXPECT_EQ(ScV4Float64Ty->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *ExtTy = VectorType::getExtendedElementVectorType(ScV8Int16Ty);
+  EXPECT_EQ(ExtTy, ScV8Int32Ty);
+  ASSERT_TRUE(ExtTy->isScalable());
+  EXPECT_EQ(ExtTy->getNumElements(), 8U);
+  EXPECT_EQ(ExtTy->getElementType()->getScalarSizeInBits(), 32U);
+
+  VectorType *TruncTy = VectorType::getTruncatedElementVectorType(ScV8Int32Ty);
+  EXPECT_EQ(TruncTy, ScV8Int16Ty);
+  ASSERT_TRUE(TruncTy->isScalable());
+  EXPECT_EQ(TruncTy->getNumElements(), 8U);
+  EXPECT_EQ(TruncTy->getElementType()->getScalarSizeInBits(), 16U);
+
+  VectorType *HalvedTy = VectorType::getHalfElementsVectorType(ScV4Int64Ty);
+  EXPECT_EQ(HalvedTy, ScV2Int64Ty);
+  ASSERT_TRUE(HalvedTy->isScalable());
+  EXPECT_EQ(HalvedTy->getNumElements(), 2U);
+  EXPECT_EQ(HalvedTy->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *DoubledTy = VectorType::getDoubleElementsVectorType(ScV4Int64Ty);
+  EXPECT_EQ(DoubledTy, ScV8Int64Ty);
+  ASSERT_TRUE(DoubledTy->isScalable());
+  EXPECT_EQ(DoubledTy->getNumElements(), 8U);
+  EXPECT_EQ(DoubledTy->getElementType()->getScalarSizeInBits(), 64U);
+
+  VectorType *ConvTy = VectorType::getInteger(ScV4Float64Ty);
+  EXPECT_EQ(ConvTy, ScV4Int64Ty);
+  ASSERT_TRUE(ConvTy->isScalable());
+  EXPECT_EQ(ConvTy->getNumElements(), 4U);
+  EXPECT_EQ(ConvTy->getElementType()->getScalarSizeInBits(), 64U);
+
+  EltCnt = ScV8Int64Ty->getElementCount();
+  EXPECT_EQ(EltCnt.Min, 8U);
+  ASSERT_TRUE(EltCnt.Scalable);
+}
+
+} // end anonymous namespace




More information about the llvm-commits mailing list