[cfe-dev] RFC: First-class Matrix type

Chandler Carruth via cfe-dev cfe-dev at lists.llvm.org
Wed Oct 17 23:14:15 PDT 2018

On Tue, Oct 16, 2018 at 11:12 AM Chris Lattner via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> On Oct 10, 2018, at 11:09 PM, Adam Nemet via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
> Hi,
> We are proposing first-class type support for a new matrix type.
> Interesting!  Here are some thoughts, I’m sorry but I haven’t read the
> responses downthread.
> This is a natural extension of the current vector type with an extra
> dimension.
> For example, this is what the IR for a matrix multiply would look like for
> a 4x4 matrix with element type float:
> %0 = load <4 x 4 x float>, <4 x 4 x float>* %a, align 16
> %1 = load <4 x 4 x float>, <4 x 4 x float>* %b, align 16
> %2 = call <4 x 4 x float> @llvm.matrix.multiply.m4_4f32.m4_4f32.m4_4f32(<4
> x 4 x float> %0, <4 x 4 x float> %1)
> store <4 x 4 x float> %2, <4 x 4 x float>* %c, align 16
> LLVM already has a pretty general vector type (arbitrary number of
> elements).  I’m aware of hardware that has rectangular vectors, e.g. nvidia
> tensor cores, Google has a proprietary in-house design with non-square
> vector registers, etc.
> Currently we support element-wise binary operations, matrix multiply,
> matrix-scalar multiply, matrix transpose, extract/insert of an element.
> Besides the regular full-matrix load and store, we also support loading and
> storing a matrix as a submatrix of a larger matrix in memory.  We are also
> planning to implement vector-extract/insert and matrix-vector multiply.
> All of these are currently implemented as intrinsics.  Where applicable we
> also plan to support these operations with native IR instructions (e.g.
> add/fadd).
> Ok.  Makes sense, I agree that supporting the existing pointwise vector
> operations makes sense.
> These are exposed in clang via builtins.  E.g. the above operations looks
> like this in C/C++:
> typedef float mf4x4_t __attribute__((matrix_type(4, 4)));
> mf4x4_t add(mf4x4_t a, mf4x4_t b) {
>   return __builtin_matrix_multiply(a, b);
> }
> I’d recommend splitting the clang discussion from the LLVM discussion,
> they are completely different tradeoffs involved.  I’ll focus on the LLVM
> IR side of things.
> ** Benefits **
> Having matrices represented as IR values allows for the usual algebraic
> and redundancy optimizations.  But most importantly, by lifting memory
> aliasing concerns, we can guarantee vectorization to target-specific
> vectors.
> Right, it is basically the same benefit as having a vector type.  You also
> get the ability to have specific alignments etc.
> I think there are several options in the design space here:
> 1. Do nothing to the type system, but just use the existing vector types
> (<16 x float> in this case) with a new set of operations.
> 2. Introduce a “matrix” concept and associated operations.
> 3. Introduce N-dimensional vector registers and associated operations.
> Personally, I’d be in favor of going with #1 followed by #3 followed
> distantly by #2.

FWIW, I strongly prefer #1 to the other options.

> The reason I’m opposed to a matrix *type* is that this is far too specific
> of a concept to put into LLVM.  We don’t even have signedness of integers
> in the type system: the instruction set is the major load bearing part of
> the IR design, and the instruction set is extensible through intrinsics.

Strongly agree.

> Arguing in favor of #1: AFAICT, you only need to add the new intrinsics to
> do matmul etc.  You could just define them to take 1D vectors but apply
> math to them that interprets them as a 2D space.  This is completely an IR
> level modeling issue, and would be a very non-invasive patch.  You’re
> literally just adding a few intrinsics.  All the pointwise operations and
> insert/extract/shuffles will “just work”.  The frontend handles mapping 2d
> indices to 1D indices.

Even better, it makes it easy to support interesting row-major and
col-major style operations w/o further diversification of the type system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20181018/a4aaf2c7/attachment.html>

More information about the cfe-dev mailing list