[llvm-dev] Questions about vscale

[Kai Wang via llvm-dev]

On Thu, Apr 16, 2020 at 10:15 PM Hanna Kruppe <hanna.kruppe at gmail.com>
wrote:

> On Mon, 13 Apr 2020 at 20:04, Renato Golin <rengolin at gmail.com> wrote:
> >
> > On Mon, 13 Apr 2020 at 18:04, Hanna Kruppe <hanna.kruppe at gmail.com>
> wrote:
> > > I don't see how the situation you mention is comparable. Legalization
> > > for e.g. <3 x i32> was not implemented at first, but as demonstrated
> > > by the fact that it *was* implemented later, there's no conceptual
> > > problem with legalizing that kind of type. You don't even have to
> > > legalize them in vector registers, three scalar registers work fine
> > > (you can even do that on the IR level).
> >
> > That was the point I was trying to make, but in my head that fused
> > with register shadowing, which derailed the point.
> >
> > To be clear, yes, "invalid" register configurations can easily usually
> > be legalised in multiple ways at lowering.
> >
> > Not all will be optimal, though, and there is where the problem lives.
> >
> > > Legalization (codegen in general) does not know if the machine
> > > code will eventually run on a chip with vector registers so small that
> > > vscale works out to 1/2, but it has to choose some legalization
> > > strategy.
> >
> > This is interesting, I had not realised that from the descriptions of
> > the problem so far. I thought it was just due to non-power-of-two
> > lengths.
> >
> > A "vector" register that is smaller than 64 bits wouldn't make much
> > sense, unless this is a DSP-type extension on very small types. In
> > those cases, every clock cycle and every instruction counts,
> > especially inside the inner loop.
> >
> > I'm struggling to see how this can be optimally executed from a
> > generic scalable code, which usually profits from the fact that vscale
> > >> 1.
>
> I can understand this kind of concern, but the specification permits
> it and this entire thread is predicated on needing to target those
> cores too. If we'd decide we're okay with LLVM-based toolchains only
> supporting hardware with e.g. VLEN >= 64 (but see [*] below) then
> there's no problem to begin with and no need for ideas like fractional
> vscale or types that can't be legalized. Alternatively, we could treat
> support for vector registers smaller than 64b as a separate ABI, like
> with soft-float ABI vs hard-float ABI.
>
> However, the current aspiration among the ISA designers and
> software/toolchain developers is different. Cores with tiny VRF are
> expected to be useful for some markets, and it's hoped that the V
> extension can "scale down" well enough to avoid the need for a second
> vector extension specifically for those cores. Personally, I have some
> doubts about how well this will work out in practice, but of course
> software and toolchain developers (including myself) would prefer to
> keep everything as portable as possible. Binary portability across
> wildly different vector register sizes is an explicit goal of the ISA,
> adding an exception to this for no good reason would be very
> unfortunate.
>

I agree. There is no such hardware configuration causing fractional vscale.
I take it into consideration due to “V” specification defining VLEN >= 32.
I also doubt it is useful for small vector register sizes.

There was some discussion in the LLVM SVE biweekly meeting last week. I
briefly summarize the discussion.

* vscale being integer could make sure the minimum size of the scalable
vector types. For example, SVE has the minimum size of vector registers be
128 bits. It has an impact on vectorizer. (I am not familiar with
vectorizer yet. I am not very sure what the impact is. It is the consensus
in the meeting. In RISC-V RVV, the vector register size, VLEN, is larger
than ELEN. It could be 32, 64, 128, etc. I think that is why RVV has such a
problem defining one type system for all hardware configurations.)
* If vscale will be fractional, take care of the special case in CodeGen
level instead of LLVM IR level. LMUL and ELEN are all known values in
compile time. It should be able to convert types in CodeGen level.

As I said, I agree with Hanna that current implementation is fine with
existing hardware. We could put the discussion aside. Even we need to deal
with fractional vscale. We could try to deal with it at CodeGen level
instead of LLVM IR level. I appreciate all of your feedback. I learned a
lot from you. Thanks a lot.

Kai


>
> [*] If we settled on requiring VLEN >= 64, we'd still face the same
> problem again if we ever want to add support for vectors elements
> larger than 64b, such as quad-precision floats or 128 bit integers. I
> don't really expect those to be commonly implemented for a long time,
> but once again: it would be great to avoid the need for a separate and
> incompatible target triple if and when such cores become relevant.
>
> > > If <vscale x 1 x i32> ends up having one element, and <vscale x 2 x
> > > i32> also has one (= 2 * 0.5) element, then that's wrong: the latter
> > > type must have twice as many elements as the former (one example where
> > > this matters: split_low / split_high / concat shuffle patterns). The
> > > second option, a vector with *zero* elements, is just as wrong if not
> > > worse.
> >
> > Right, that was the idea behind vscale from the beginning. I don't
> > know how many elements either has, but I know the latter has twice as
> > many as the former.
> >
> > I see why you would want half-length, because that truth still holds:
> > the latter has twice as many halves as the former.
> >
> > But how do you handle the last half? Do you ignore? Do you load /
> > store half? Do you always mask it out? Do you fuse with the next
> > iterations' first half?
> >
> > If the semantics is not clear on how the back-ends are supposed to use
> > that extra half, then extending the IR in such a way can make it very
> > hard for generic optimisations to understand anything about the
> > ranges, validity of operations, alignment, masks, undefined behaviour,
> > etc.
> >
> > > It's not that a correct legalization exists but it's too annoying to
> > > implement, or that one might exist but I'm too lazy to work it out.
> >
> > I never meant to imply that. Apologies if that's what came through.
>
> Oh no, not at all! Sorry for the confusion, I should avoid rhetoric
> that can create this impression.
>
> > > We're also not running in a limitation or oddity of the RISC-V vector
> > > ISA in particular. It's simply that, if you set vscale == 0.5, then by
> > > the way scalable vector types work (vscale * const elements), some
> > > vector types that can be written in the IR would need to have a
> > > fractional number of elements to be consistent with the other scalable
> > > vector types. As that is not possible (not even conceptually),
> > > whatever code you emit to try to legalize that type will end up being
> > > wrong in some respect.
> >
> > Honestly, I'm running out of breath in this discussion. :)
> >
> > I don't know a lot about SVE and even less about RISC-V, so I'll leave
> > the more in-depth technical discussions for Florian/Sander and others
> > to chime in.
>
> Fair, thanks for the discussion so far :)
>
> Best regards
> Hanna
>
> > > So if we'd decide to support fractional vscale, we can't say these
> > > types are "illegal". In LLVM parlance, illegal types can be used in
> > > LLVM IR and targets aspire to turn them into something that works
> > > correctly, even if it's very inefficient. Sometimes a legalization is
> > > unimplemented or buggy, but these problems can be patched and this has
> > > often happened in the past. With fractional vscale, the situation is
> > > quite different: nobody will ever be able to use certain scalable
> > > vector types on the target in question, because they can't be
> > > legalized even in principle.
> >
> > I have not spent the time you guys have on this, but if I understood
> > your problem correctly, I too can't think of a way to represent this
> > in non-fractional ways.
> >
> > I'm not saying this is a good idea, and I think you're not saying it
> > is either, but perhaps the only idea.
> >
> > If that's the case, then I have proposed to use a different
> > flag/integer to mean half-scale instead of floating points, and
> > hopefully that can be transparent to the rest of scalable vector code.
> >
> > But I'd really like to get other people's point of view, as I'm not
> > confident on my appraisal.
> >
> > > I hope this lengthy explanation help you see where I'm coming from.
> >
> > It did, thanks!
> >
> > --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200420/7060664e/attachment.html>