[all-commits] [llvm/llvm-project] 013cf4: [CIR][AArch64] Support BF16 Neon types and lower v...

[Jiahao Guo via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 013cf4fd1fefb11223c17c6a47b6b7d635323789
      https://github.com/llvm/llvm-project/commit/013cf4fd1fefb11223c17c6a47b6b7d635323789
  Author: Jiahao Guo <eoonguo at gmail.com>
  Date:   2026-03-27 (Fri, 27 Mar 2026)

  Changed paths:
    M clang/lib/CIR/CodeGen/CIRGenBuiltinAArch64.cpp
    M clang/test/CodeGen/AArch64/bf16-getset-intrinsics.c
    M clang/test/CodeGen/AArch64/neon/bf16-getset.c

  Log Message:
  -----------
  [CIR][AArch64] Support BF16 Neon types and lower vdup lane builtins (#187460)

Part of https://github.com/llvm/llvm-project/issues/185382.

Lower:
-
[vdup_n_bf16](https://developer.arm.com/architectures/instruction-sets/intrinsics/vdup_n_bf16)
-
[vdupq_n_bf16](https://developer.arm.com/architectures/instruction-sets/intrinsics/vdupq_n_bf16)
-
[vdup_lane_bf16](https://developer.arm.com/architectures/instruction-sets/intrinsics/vdup_lane_bf16)
-
[vdupq_lane_bf16](https://developer.arm.com/architectures/instruction-sets/intrinsics/vdupq_lane_bf16)
-
[vdup_laneq_bf16](https://developer.arm.com/architectures/instruction-sets/intrinsics/vdup_laneq_bf16)
-
[vdupq_laneq_bf16](https://developer.arm.com/architectures/instruction-sets/intrinsics/vdupq_laneq_bf16)

and add tests in
[bf16-getset.c](https://github.com/llvm/llvm-project/blob/main/clang/test/CodeGen/AArch64/neon/bf16-getset.c).

## Approach

### `vdup_n_bf16` / `vdupq_n_bf16`

These are not NEON builtins — they are regular `always_inline` functions
defined in `arm_neon.h` that expand to vector aggregate initialization
(`{v, v, v, v}`), so they work through the existing generic vector
codegen path without requiring any builtin-specific handling. I just
added CHECK lines in `bf16-getset.c` to verify the existing output is
correct.

### `vdup_lane_bf16` / `vdupq_lane_bf16` / `vdup_laneq_bf16` /
`vdupq_laneq_bf16`

These are mapped (via `NEONEquivalentIntrinsicMap`) to the generic
`splat_lane_v` / `splatq_lane_v` / `splat_laneq_v` / `splatq_laneq_v`
builtins, which are handled in `emitCommonNeonBuiltinExpr`.

I followed the approach used in both the OG codegen (`ARM.cpp`) and the
[clangir incubator](https://github.com/nicovank/clangir):

**OG codegen in `ARM.cpp`:**

```cpp
switch (BuiltinID) {
  default: break;
  case NEON::BI__builtin_neon_splat_lane_v:
  case NEON::BI__builtin_neon_splat_laneq_v:
  case NEON::BI__builtin_neon_splatq_lane_v:
  case NEON::BI__builtin_neon_splatq_laneq_v: {
    auto NumElements = VTy->getElementCount();
    if (BuiltinID == NEON::BI__builtin_neon_splatq_lane_v)
      NumElements = NumElements * 2;
    if (BuiltinID == NEON::BI__builtin_neon_splat_laneq_v)
      NumElements = NumElements.divideCoefficientBy(2);

    Ops[0] = Builder.CreateBitCast(Ops[0], VTy);
    return EmitNeonSplat(Ops[0], cast<ConstantInt>(Ops[1]), NumElements);
  }
```

**clangir incubator in `CIRGenBuiltinAArch64.cpp`:**

```cpp
  case NEON::BI__builtin_neon_splat_lane_v:
  case NEON::BI__builtin_neon_splat_laneq_v:
  case NEON::BI__builtin_neon_splatq_lane_v:
  case NEON::BI__builtin_neon_splatq_laneq_v: {
    uint64_t numElements = vTy.getSize();
    if (builtinID == NEON::BI__builtin_neon_splatq_lane_v)
      numElements = numElements << 1;
    if (builtinID == NEON::BI__builtin_neon_splat_laneq_v)
      numElements = numElements >> 1;
    ops[0] = builder.createBitcast(ops[0], vTy);
    return emitNeonSplat(builder, getLoc(e->getExprLoc()), ops[0], ops[1],
                         numElements);
  }
```

The call site for `splat_lane_v` already existed in
`emitCommonNeonBuiltinExpr`, but had two issues:

1. **`emitNeonSplat` was called but never defined.** I added two helper
functions (ported from the clangir incubator): `getIntValueFromConstOp`
to extract the integer lane index from a CIR constant, and
`emitNeonSplat` to build a splat shuffle mask and perform a
`cir.vec.shuffle`.

2. **The call site used `getLoc(e->getExprLoc())`, which is invalid**
because `emitCommonNeonBuiltinExpr` is a static free function, not a
`CIRGenFunction` member. Fixed to use `cgf.getBuilder()` and the
pre-computed `loc` variable.

Additionally, I found that `NeonTypeFlags::BFloat16` and
`NeonTypeFlags::Float16` were unhandled in `getNeonType`, which would
cause the vector type to be unresolved for bf16/f16 intrinsics. I added
the handling following the same pattern as the OG codegen:

```cpp
  case NeonTypeFlags::BFloat16:
    if (allowBFloatArgsAndRet)
      return cir::VectorType::get(cgf->getCIRGenModule().bFloat16Ty, v1Ty ? 1 : (4 << isQuad));
    return cir::VectorType::get(cgf->uInt16Ty, v1Ty ? 1 : (4 << isQuad));
  case NeonTypeFlags::Float16:
    if (hasLegalHalfType)
      return cir::VectorType::get(cgf->getCIRGenModule().fP16Ty, v1Ty ? 1 : (4 << isQuad));
    return cir::VectorType::get(cgf->uInt16Ty, v1Ty ? 1 : (4 << isQuad));
```

When `allowBFloatArgsAndRet` is true, we use the native `cir::BF16Type`;
otherwise we fall back to `u16i`. The same logic applies to `Float16`
with `hasLegalHalfType`.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications