[all-commits] [llvm/llvm-project] 4fd4ec: [AArch64][SVE] Add support for trunc to <vscale x ...

Fri Jul 31 08:35:54 PDT 2020

  Branch: refs/heads/release/11.x
  Home:   https://github.com/llvm/llvm-project
  Commit: 4fd4ec63813fd5b22d81adb6e201cb16ccf72b69
      https://github.com/llvm/llvm-project/commit/4fd4ec63813fd5b22d81adb6e201cb16ccf72b69
  Author: Eli Friedman <efriedma at quicinc.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
    M llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
    M llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
    M llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
    M llvm/test/CodeGen/AArch64/sve-trunc.ll

  Log Message:
  -----------
  [AArch64][SVE] Add support for trunc to <vscale x N x i1>.

This isn't a natively supported operation, so convert it to a
mask+compare.

In addition to the operation itself, fix up some surrounding stuff to
make the testcase work: we need concat_vectors on i1 vectors, we need
legalization of i1 vector truncates, and we need to fix up all the
relevant uses of getVectorNumElements().

Differential Revision: https://reviews.llvm.org/D83811

(cherry picked from commit b8f765a1e17f8d212ab1cd8f630d35adc7495556)

  Commit: 328269f3834d793bd4a7287d4344aa266d6641b9
      https://github.com/llvm/llvm-project/commit/328269f3834d793bd4a7287d4344aa266d6641b9
  Author: Sander de Smalen <sander.desmalen at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp
    M llvm/test/CodeGen/AArch64/sve-calling-convention.ll
    M llvm/test/CodeGen/AArch64/sve-trunc.ll

  Log Message:
  -----------
  [AArch64][SVE] Fix PCS for functions taking/returning scalable types.

The default calling convention needs to save/restore the SVE callee
saves according to the SVE PCS when the function takes or returns
scalable types, even when the `aarch64_sve_vector_pcs` CC is not
specified for the function.

Reviewers: efriedma, paulwalker-arm, david-arm, rengolin

Reviewed By: paulwalker-arm

Differential Revision: https://reviews.llvm.org/D84041

(cherry picked from commit 9bacf1588583014538a0217add18f370acb95788)

  Commit: fd2d5a0c4cdc9ccb0b88f264ae452e3a0e8dcc09
      https://github.com/llvm/llvm-project/commit/fd2d5a0c4cdc9ccb0b88f264ae452e3a0e8dcc09
  Author: Sander de Smalen <sander.desmalen at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp
    A llvm/test/CodeGen/AArch64/framelayout-scavengingslot.mir
    A llvm/test/CodeGen/AArch64/framelayout-sve-basepointer.mir
    A llvm/test/CodeGen/AArch64/framelayout-sve-scavengingslot.mir
    M llvm/test/CodeGen/AArch64/framelayout-sve.mir

  Log Message:
  -----------
  [AArch64][SVE] Correctly allocate scavenging slot in presence of SVE.

This patch addresses two issues:

* Forces the availability of the base-pointer (x19) when the frame has
  both scalable vectors and variable-length arrays. Otherwise it will
  be expensive to access non-SVE locals.

* In presence of SVE stack objects, it will allocate the emergency
  scavenging slot close to the SP, so that they can be accessed from
  the SP or BP if available. If accessed from the frame-pointer, it will
  otherwise need an extra register to access the scavenging slot because
  of mixed scalable/non-scalable addressing modes.

Reviewers: efriedma, ostannard, cameron.mcinally, rengolin, david-arm

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D70174

(cherry picked from commit bef56f7fe2382ed1476aa67a55626b364635b44e)

  Commit: 48eb1aa387eb1d356632b82efaf6438d1fcb6640
      https://github.com/llvm/llvm-project/commit/48eb1aa387eb1d356632b82efaf6438d1fcb6640
  Author: Eli Friedman <efriedma at quicinc.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
    A llvm/test/CodeGen/AArch64/sve-copy-zprpair.mir

  Log Message:
  -----------
  [AArch64][SVE] Teach copyPhysReg to copy ZPR2/3/4.

It's sort of tricky to hit this in practice, but not impossible. I have
a synthetic C testcase if anyone is interested.

The implementation is identical to the equivalent NEON register copies.

Differential Revision: https://reviews.llvm.org/D84373

(cherry picked from commit 993c1a3219a8ae69f1d700183bf174d75f3815d4)

  Commit: 967b84c7a7e2a39d01ab4266bf5eac8c2de98ce5
      https://github.com/llvm/llvm-project/commit/967b84c7a7e2a39d01ab4266bf5eac8c2de98ce5
  Author: Sander de Smalen <sander.desmalen at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64FrameLowering.cpp
    M llvm/test/CodeGen/AArch64/framelayout-sve.mir

  Log Message:
  -----------
  [AArch64][SVE] Don't support fixedStack for SVE objects.

Fixed stack objects are preallocated and defined to be allocated before
any of the regular stack objects. These are normally used to model stack
arguments.

The AAPCS does not support passing SVE registers on the stack by value
(only by reference). The current layout also doesn't place them before
all stack objects, but rather before all SVE objects. Removing this
simplifies the code that emits the allocation/deallocation
around callee-saved registers (D84042).

This patch also removes all uses of fixedStack from from
framelayout-sve.mir, where this was used purely for testing purposes.

Reviewers: paulwalker-arm, efriedma, rengolin

Reviewed By: paulwalker-arm

Differential Revision: https://reviews.llvm.org/D84538

(cherry picked from commit 54492a5843a34684ce21ae201dd8ca3e509288fd)

  Commit: 5596693504af263d53d6676ec6f0f4669ac706b0
      https://github.com/llvm/llvm-project/commit/5596693504af263d53d6676ec6f0f4669ac706b0
  Author: Sander de Smalen <sander.desmalen at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64FrameLowering.cpp
    M llvm/test/CodeGen/AArch64/framelayout-sve.mir
    M llvm/test/CodeGen/AArch64/sve-trunc.ll

  Log Message:
  -----------
  [AArch64][SVE] Don't align the last SVE callee save.

Instead of aligning the last callee-saved-register slot to the stack
alignment (16 bytes), just align the SVE callee-saved block. This also
simplifies the code that allocates space for the callee-saves.

This change is needed to make sure the offset to which the callee-saved
register is spilled, corresponds to the offset used for e.g. unwind call
frame instructions.

Reviewers: efriedma, paulwalker-arm, david-arm, rengolin

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D84042

(cherry picked from commit 26b4ef3694973ea2fa656d3d3a7f67f16f135654)

  Commit: 13fb3d3007f0c2ddfc14a1819def5bec1533a0b0
      https://github.com/llvm/llvm-project/commit/13fb3d3007f0c2ddfc14a1819def5bec1533a0b0
  Author: Sander de Smalen <sander.desmalen at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64FrameLowering.cpp
    M llvm/test/CodeGen/AArch64/framelayout-sve.mir

  Log Message:
  -----------
  [AArch64][SVE] Fix epilogue for SVE when the stack is realigned.

While deallocating the stackframe, the offset used to reload the
callee-saved registers was not pointing to the SVE callee-saves,
but rather to the whole SVE area.

   +--------------+
   | GRP callee   |
   |     saves    |
   +--------------+ <- FP
   | SVE callee   |
   |     saves    |
   +--------------+ <- Should restore SVE callee saves from here
   |  SVE Spills  |
   |  and Locals  |
   +--------------+ <- instead of from here.
   |              |
   :              :
   |              |
   +--------------+ <- SP

Reviewed By: paulwalker-arm

Differential Revision: https://reviews.llvm.org/D84539

(cherry picked from commit cda2eb3ad2bbe923e74d6eb083af196a0622d800)

  Commit: a3532c58be5c3a4107549c2462613be76507fe55
      https://github.com/llvm/llvm-project/commit/a3532c58be5c3a4107549c2462613be76507fe55
  Author: David Sherwood <david.sherwood at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/include/llvm/CodeGen/TargetFrameLowering.h
    M llvm/lib/CodeGen/LocalStackSlotAllocation.cpp
    M llvm/lib/Target/AArch64/AArch64FrameLowering.h
    A llvm/test/CodeGen/AArch64/sve-localstackalloc.mir

  Log Message:
  -----------
  [SVE] Don't use LocalStackAllocation for SVE objects

I have introduced a new TargetFrameLowering query function:

  isStackIdSafeForLocalArea

that queries whether or not it is safe for objects of a given stack
id to be bundled into the local area. The default behaviour is to
always bundle regardless of the stack id, however for AArch64 this is
overriden so that it's only safe for fixed-size stack objects.
There is future work here to extend this algorithm for multiple local
areas so that SVE stack objects can be bundled together and accessed
from their own virtual base-pointer.

Differential Revision: https://reviews.llvm.org/D83859

(cherry picked from commit 14bc85e0ebb6c00c1672158ab6a692bfbb11e1cc)

  Commit: 75e5d4f42ad9556f5d53bb2984a23082d6c6a830
      https://github.com/llvm/llvm-project/commit/75e5d4f42ad9556f5d53bb2984a23082d6c6a830
  Author: David Sherwood <david.sherwood at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

  Log Message:
  -----------
  [CodeGen] Remove calls to getVectorNumElements in DAGTypeLegalizer::SplitVecOp_EXTRACT_SUBVECTOR

In DAGTypeLegalizer::SplitVecOp_EXTRACT_SUBVECTOR I have replaced
calls to getVectorNumElements with getVectorMinNumElements, since
this code path works for both fixed and scalable vector types. For
scalable vectors the index will be multiplied by VSCALE.

Fixes warnings in this test:

  sve-sext-zext.ll

Differential revision: https://reviews.llvm.org/D83198

(cherry picked from commit 5d84eafc6b86a42e261af8d753c3a823e0e7c67e)

  Commit: 07a7044b805a422469041928c7c6ee55bcdda2a4
      https://github.com/llvm/llvm-project/commit/07a7044b805a422469041928c7c6ee55bcdda2a4
  Author: David Sherwood <david.sherwood at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/test/CodeGen/AArch64/sve-sext-zext.ll

  Log Message:
  -----------
  [SVE] Add checks for no warnings in CodeGen/AArch64/sve-sext-zext.ll

Previous patches fixed up all the warnings in this test:

  llvm/test/CodeGen/AArch64/sve-sext-zext.ll

and this change simply checks that no new warnings are added in future.

Differential revision: https://reviews.llvm.org/D83205

(cherry picked from commit f43b5c7a76ab83dcc80e6769d41d5c4b761312b1)

  Commit: bec306442de89c71c2268e7e2629b4d454895a56
      https://github.com/llvm/llvm-project/commit/bec306442de89c71c2268e7e2629b4d454895a56
  Author: David Sherwood <david.sherwood at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
    M llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp
    M llvm/test/CodeGen/AArch64/sve-intrinsics-int-arith.ll

  Log Message:
  -----------
  [SVE][CodeGen] Add simple integer add tests for SVE tuple types

I have added tests to:

  CodeGen/AArch64/sve-intrinsics-int-arith.ll

for doing simple integer add operations on tuple types. Since these
tests introduced new warnings due to incorrect use of
getVectorNumElements() I have also fixed up these warnings in the
same patch. These fixes are:

1. In narrowExtractedVectorBinOp I have changed the code to bail out
early for scalable vector types, since we've not yet hit a case that
proves the optimisations are profitable for scalable vectors.
2. In DAGTypeLegalizer::WidenVecRes_CONCAT_VECTORS I have replaced
calls to getVectorNumElements with getVectorMinNumElements in cases
that work with scalable vectors. For the other cases I have added
asserts that the vector is not scalable because we should not be
using shuffle vectors and build vectors in such cases.

Differential revision: https://reviews.llvm.org/D84016

(cherry picked from commit 207877175944656bd9b52d36f391a092854572be)

  Commit: 16a68abcebc0c09edaa03bde11e0f452cfd4abdf
      https://github.com/llvm/llvm-project/commit/16a68abcebc0c09edaa03bde11e0f452cfd4abdf
  Author: David Sherwood <david.sherwood at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
    M llvm/test/Transforms/SLPVectorizer/AArch64/scalable-vector.ll

  Log Message:
  -----------
  [SVE] Don't consider scalable vector types in SLPVectorizerPass::vectorizeChainsInBlock

In vectorizeChainsInBlock we try to collect chains of PHI nodes
that have the same element type, but the code is relying upon
the implicit conversion from TypeSize -> uint64_t. For now, I have
modified the code to ignore PHI nodes with scalable types.

Differential Revision: https://reviews.llvm.org/D83542

(cherry picked from commit 9ad7c980bb47edd7db8f8db828b487cc7dfc9921)

  Commit: 5583444d188015fbcf97d16c946b2617af81698a
      https://github.com/llvm/llvm-project/commit/5583444d188015fbcf97d16c946b2617af81698a
  Author: David Sherwood <david.sherwood at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
    M llvm/test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll

  Log Message:
  -----------
  [SVE][CodeGen] At -O0 fallback to DAG ISel when translating alloca with scalable types

When building code at -O0 We weren't falling back to DAG ISel correctly
when encountering alloca instructions with scalable vector types. This
is because the alloca has no operands that are scalable. I've fixed this by
adding a check in AArch64ISelLowering::fallBackToDAGISel for alloca
instructions with scalable types.

Differential Revision: https://reviews.llvm.org/D84746

(cherry picked from commit 23ad660b5d34930b2b5362f1bba63daee78f6aa4)

  Commit: 6b66be512110acb2dfdab00d9755d86e185f1e3d
      https://github.com/llvm/llvm-project/commit/6b66be512110acb2dfdab00d9755d86e185f1e3d
  Author: Francesco Petrogalli <francesco.petrogalli at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
    M llvm/lib/Target/AArch64/AArch64InstrFormats.td
    M llvm/lib/Target/AArch64/SVEInstrFormats.td
    A llvm/test/CodeGen/AArch64/sve-intrinsics-ld1ro-addressing-mode-reg-imm.ll

  Log Message:
  -----------
  [llvm][sve] Reg + Imm addressing mode for ld1ro.

Reviewers: kmclaughlin, efriedma, sdesmalen

Subscribers: tschuett, hiraditya, psnobl, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D83357

(cherry picked from commit 809600d6642773f71245f76995dab355effc73af)

  Commit: db306412bf65f4b6fa4314dd5611752448bbc80c
      https://github.com/llvm/llvm-project/commit/db306412bf65f4b6fa4314dd5611752448bbc80c
  Author: Francesco Petrogalli <francesco.petrogalli at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64ISelDAGToDAG.cpp

  Log Message:
  -----------
  [NFC][AArch64] Replace some template methods/invocations...

...with the non-template version, as the template version might
increase the size of the compiler build.

Methods affected:

1.`findAddrModeSVELoadStore`
2. `SelectPredicatedStore`

Also, remove the `const` qualifier from the `unsigned` parameters of
the methods to conform with other similar methods in the class.

(cherry picked from commit dbeb184b7f54db2d3ef20ac153b1c77f81cf0b99)

  Commit: f3fa634264e3504ab684dfd29307594488878f14
      https://github.com/llvm/llvm-project/commit/f3fa634264e3504ab684dfd29307594488878f14
  Author: Francesco Petrogalli <francesco.petrogalli at arm.com>
  Date:   2020-07-31 (Fri, 31 Jul 2020)

  Changed paths:
    M llvm/lib/Target/AArch64/AArch64ISelDAGToDAG.cpp
    A llvm/test/CodeGen/AArch64/sve-intrinsics-ldN-reg+imm-addr-mode.ll
    A llvm/test/CodeGen/AArch64/sve-intrinsics-ldN-reg+reg-addr-mode.ll

  Log Message:
  -----------
  [llvm][CodeGen] Addressing modes for SVE ldN.

Reviewers: c-rhodes, efriedma, sdesmalen

Subscribers: huihuiz, tschuett, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77251

(cherry picked from commit adb28e0fb2b0e97ea9dce422c09b36979cf7cd2f)

Compare: https://github.com/llvm/llvm-project/compare/3286126de175...f3fa634264e3