[llvm-branch-commits] [flang] [mlir] [Flang][mlir][OpenMP] Translate omp.declare_simd to LLVM IR (PR #187767)
via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Fri Mar 20 11:50:30 PDT 2026
llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-mlir-llvm
@llvm/pr-subscribers-flang-fir-hlfir
Author: Chi-Chun, Chen (chichunchen)
<details>
<summary>Changes</summary>
Translate omp.declare_simd operations to LLVM IR by computing Vector
Function ABI (VFABI) mangled names and attaching them as function
attributes. This reuse's function parameter mangling and codegen logic
in OpenMPIRBuilder which was extracted form Clang [2][3].
For each omp.declare_simd, lowering computes:
- ParamAttrs: one entry per function argument, classifying it as
Vector / Uniform / Linear (+ step or var-stride) / Aligned.
- Branch kind: Undefined / Inbranch / Notinbranch.
- VLEN: either from simdlen() or derived from ISA-specific rules.
x86 (SSE/AVX/AVX2/AVX-512):
Emits mangled names following the x86 Vector ABI [1]:
```
_ZGV<ISA><Mask><VLEN><ParamAttrs>_<name>
```
where ISA is b (SSE), c (AVX), d (AVX2), e (AVX-512). Without an
explicit simdlen, VLEN is computed from the Characteristic Data Type
(CDT) and the vector register width for each ISA.
AArch64 (AdvSIMD / SVE):
Emits mangled names following the AAVFABI [4]:
```
_ZGV<ISA><Mask><VLEN><ParamAttrs>_<name>
```
where ISA is n (AdvSIMD/Neon) or s (SVE). Without an explicit simdlen,
AdvSIMD VLEN is derived from the Narrowest Data Size (NDS) of the
function signature. The implementation computes Maps To Vector (MTV),
Pass By Value (PBV), Lane Size (LS), NDS, and Widest Data Size (WDS)
per the AAVFABI spec. WDS and architectural limits (128-2048 bit,
128-bit aligned) are used to validate explicit simdlen values, with
diagnostics emitted for invalid cases.
Also adds:
- Linear modifier (val/ref/uval) support in parameter attribute
lowering, mapping omp::LinearModifier to the appropriate
DeclareSimdKindTy.
- An arg_types attribute on DeclareSimdOp to recover language-level
pointee type information for opaque !llvm.ptr parameters, enabling
correct LS/NDS/WDS computation.
[1] https://sourceware.org/glibc/wiki/libmvec?action=AttachFile&do=view&target=VectorABI.txt
[2] https://github.com/llvm/llvm-project/commit/c7a82b41a706728ce7c212b5bc40c74d1cce53c7
[3] https://github.com/llvm/llvm-project/commit/a0a2264ef757f8383c6b283b7ad80b33d5d52f13
[4] https://github.com/ARM-software/abi-aa/tree/main/vfabia64
Assisted by copilot with claude model
---
Patch is 81.68 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/187767.diff
15 Files Affected:
- (modified) flang/lib/Lower/OpenMP/ClauseProcessor.cpp (+49-6)
- (modified) flang/lib/Lower/OpenMP/OpenMP.cpp (+31-1)
- (modified) flang/test/Lower/OpenMP/declare-simd.f90 (+16-12)
- (modified) mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td (+8)
- (modified) mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp (+9-1)
- (modified) mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp (+391)
- (modified) mlir/test/Dialect/OpenMP/invalid.mlir (+8)
- (modified) mlir/test/Dialect/OpenMP/ops.mlir (+16)
- (added) mlir/test/Target/LLVMIR/openmp-declare-simd-aarch64-01.mlir (+314)
- (added) mlir/test/Target/LLVMIR/openmp-declare-simd-aarch64-02.mlir (+39)
- (added) mlir/test/Target/LLVMIR/openmp-declare-simd-aarch64-nds.mlir (+86)
- (added) mlir/test/Target/LLVMIR/openmp-declare-simd-aarch64-sve.mlir (+37)
- (added) mlir/test/Target/LLVMIR/openmp-declare-simd-aarch64-warnings.mlir (+74)
- (added) mlir/test/Target/LLVMIR/openmp-declare-simd-aarch64-wds.mlir (+108)
- (added) mlir/test/Target/LLVMIR/openmp-declare-simd-x86.mlir (+610)
``````````diff
diff --git a/flang/lib/Lower/OpenMP/ClauseProcessor.cpp b/flang/lib/Lower/OpenMP/ClauseProcessor.cpp
index c4486b8719b70..93abb10e6ba24 100644
--- a/flang/lib/Lower/OpenMP/ClauseProcessor.cpp
+++ b/flang/lib/Lower/OpenMP/ClauseProcessor.cpp
@@ -1520,6 +1520,9 @@ bool ClauseProcessor::processLinear(mlir::omp::LinearClauseOps &result,
}
}
+ fir::FirOpBuilder &firOpBuilder = converter.getFirOpBuilder();
+ mlir::Location currentLocation = converter.getCurrentLocation();
+
for (const omp::Object &object : objects) {
semantics::Symbol *sym = object.sym();
const mlir::Value variable = converter.getSymbolAddress(*sym);
@@ -1527,20 +1530,17 @@ bool ClauseProcessor::processLinear(mlir::omp::LinearClauseOps &result,
mlir::Type ty = converter.genType(*sym);
typeAttrs.push_back(mlir::TypeAttr::get(ty));
+ mlir::Value stepOperand;
if (auto &mod =
std::get<std::optional<omp::clause::Linear::StepComplexModifier>>(
clause.t)) {
- mlir::Value operand =
+ stepOperand =
fir::getBase(converter.genExprValue(toEvExpr(*mod), stmtCtx));
- result.linearStepVars.append(objects.size(), operand);
} else {
// If nothing is present, add the default step of 1.
- fir::FirOpBuilder &firOpBuilder = converter.getFirOpBuilder();
- mlir::Location currentLocation = converter.getCurrentLocation();
mlir::Type integerTy = ty.isInteger() ? ty : firOpBuilder.getI32Type();
- mlir::Value operand =
+ stepOperand =
firOpBuilder.createIntegerConstant(currentLocation, integerTy, 1);
- result.linearStepVars.append(objects.size(), operand);
}
// Determine the linear modifier:
@@ -1569,6 +1569,49 @@ bool ClauseProcessor::processLinear(mlir::omp::LinearClauseOps &result,
linearMod = isDeclareSimd ? getDeclareSimdDefaultMod(*sym)
: mlir::omp::LinearModifier::val;
+ // For declare simd, rescale constant linear steps from source-level
+ // element counts to byte strides for parameters that will use Linear
+ // or LinearRef ABI kind. This matches Clang's rescaling behavior in
+ // CGOpenMPRuntime.cpp. Val and UVal kinds are not rescaled because
+ // they describe value changes, not pointer strides.
+ if (isDeclareSimd) {
+ bool isRefLike = false;
+ const auto &ultimate = sym->GetUltimate();
+ if (semantics::IsPointer(ultimate))
+ isRefLike = true;
+ else if (const auto *obj =
+ ultimate
+ .detailsIf<Fortran::semantics::ObjectEntityDetails>())
+ if (obj->isDummy() && !semantics::IsValue(ultimate))
+ isRefLike = true;
+
+ // Rescale for ref modifier or no modifier on a reference-like param.
+ // val and uval are not rescaled.
+ bool needsRescale =
+ isRefLike &&
+ (!linearMod || *linearMod == mlir::omp::LinearModifier::ref);
+ if (needsRescale) {
+ mlir::Type elemTy = fir::getFortranElementType(ty);
+ if (elemTy.isIntOrFloat()) {
+ unsigned elemSizeBytes = elemTy.getIntOrFloatBitWidth() / 8;
+ if (elemSizeBytes > 1) {
+ // Only rescale constant strides.
+ if (auto cstOp =
+ stepOperand.getDefiningOp<mlir::arith::ConstantOp>()) {
+ if (auto intAttr =
+ mlir::dyn_cast<mlir::IntegerAttr>(cstOp.getValue())) {
+ int64_t rescaled = intAttr.getInt() * elemSizeBytes;
+ stepOperand = firOpBuilder.createIntegerConstant(
+ currentLocation, stepOperand.getType(), rescaled);
+ }
+ }
+ }
+ }
+ }
+ }
+
+ result.linearStepVars.push_back(stepOperand);
+
if (linearMod)
linearModAttrs.push_back(mlir::omp::LinearModifierAttr::get(
&converter.getMLIRContext(), *linearMod));
diff --git a/flang/lib/Lower/OpenMP/OpenMP.cpp b/flang/lib/Lower/OpenMP/OpenMP.cpp
index 152541e83372b..d0a6725a6ab8e 100644
--- a/flang/lib/Lower/OpenMP/OpenMP.cpp
+++ b/flang/lib/Lower/OpenMP/OpenMP.cpp
@@ -4043,7 +4043,37 @@ genOMP(lower::AbstractConverter &converter, lower::SymMap &symTable,
cp.processSimdlen(clauseOps);
cp.processUniform(clauseOps);
- mlir::omp::DeclareSimdOp::create(converter.getFirOpBuilder(), loc, clauseOps);
+ auto declareSimdOp = mlir::omp::DeclareSimdOp::create(
+ converter.getFirOpBuilder(), loc, clauseOps);
+
+ // Record the scalar element types of all function arguments so that
+ // the OpenMPToLLVMIRTranslation can recover pointee-type information
+ // lost in opaque pointers for correct LS / NDS / WDS computation.
+ // We strip FIR wrappers (box, heap, ref, array) to get the plain scalar
+ // type (e.g. i32, f64) that survives FIR-to-LLVM type conversion unchanged.
+ if (auto *owningProc = eval.getOwningProcedure();
+ owningProc && !owningProc->isMainProgram()) {
+ const auto &subpSym = owningProc->getSubprogramSymbol();
+ if (auto *details =
+ subpSym.GetUltimate()
+ .detailsIf<Fortran::semantics::SubprogramDetails>()) {
+ llvm::SmallVector<mlir::Attribute> argTypeAttrs;
+ for (const auto *arg : details->dummyArgs()) {
+ if (arg) {
+ mlir::Type ty = converter.genType(*arg);
+ // Unwrap FIR container types to get the scalar element type.
+ ty = fir::getFortranElementType(ty);
+ argTypeAttrs.push_back(mlir::TypeAttr::get(ty));
+ } else {
+ argTypeAttrs.push_back(mlir::TypeAttr::get(
+ mlir::NoneType::get(&converter.getMLIRContext())));
+ }
+ }
+ if (!argTypeAttrs.empty())
+ declareSimdOp.setArgTypesAttr(
+ mlir::ArrayAttr::get(&converter.getMLIRContext(), argTypeAttrs));
+ }
+ }
}
static void genOpenMPDeclareMapperImpl(
diff --git a/flang/test/Lower/OpenMP/declare-simd.f90 b/flang/test/Lower/OpenMP/declare-simd.f90
index b2c4592ad4e8a..429dd3409913f 100644
--- a/flang/test/Lower/OpenMP/declare-simd.f90
+++ b/flang/test/Lower/OpenMP/declare-simd.f90
@@ -38,7 +38,8 @@ end subroutine declare_simd_aligned
! CHECK: %[[X_A:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE_A]] arg 1 {{.*pointer.*}} : (!fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, !fir.dscope) -> (!fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>)
! CHECK: %[[Y_A:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE_A]] arg 2 {{.*pointer.*}} : (!fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, !fir.dscope) -> (!fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>)
! CHECK: omp.declare_simd aligned(%[[X_A]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>> -> 64 : i64,
-! CHECK-SAME: %[[Y_A]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>> -> 64 : i64){{$}}
+! CHECK-SAME: %[[Y_A]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>> -> 64 : i64)
+! CHECK-SAME: {arg_types = [f64, f64, i32, i32]}{{$}}
! CHECK: return
subroutine declare_simd_linear(x, y, n, i)
@@ -56,8 +57,9 @@ end subroutine declare_simd_linear
! CHECK-LABEL: func.func @_QPdeclare_simd_linear(
! CHECK: %[[SCOPE:.*]] = fir.dummy_scope : !fir.dscope
! CHECK: %[[I:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE]] arg 4 {{.*}} : (!fir.ref<i32>, !fir.dscope) -> (!fir.ref<i32>, !fir.ref<i32>)
-! CHECK: %[[C1:.*]] = arith.constant 1 : i32
-! CHECK: omp.declare_simd linear(ref(%[[I]]#0 : !fir.ref<i32> = %[[C1]] : i32)) {linear_var_types = [i32]}{{$}}
+! CHECK: %[[C4:.*]] = arith.constant 4 : i32
+! CHECK: omp.declare_simd linear(ref(%[[I]]#0 : !fir.ref<i32> = %[[C4]] : i32))
+! CHECK-SAME: {arg_types = [f64, f64, i32, i32], linear_var_types = [i32]}{{$}}
! CHECK: return
subroutine declare_simd_simdlen(x, y, n, i)
@@ -70,7 +72,8 @@ end subroutine declare_simd_simdlen
! CHECK-LABEL: func.func @_QPdeclare_simd_simdlen(
! CHECK: %[[SCOPE_S:.*]] = fir.dummy_scope : !fir.dscope
-! CHECK: omp.declare_simd{{.*}}simdlen(8){{$}}
+! CHECK: omp.declare_simd{{.*}}simdlen(8)
+! CHECK-SAME: {arg_types = [f32, f32, i32, i32]}{{$}}
! CHECK-NEXT: return
subroutine declare_simd_uniform(x, y, n, i)
@@ -100,6 +103,7 @@ end subroutine declare_simd_uniform
! CHECK: %[[XDECL:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE]] arg 1
! CHECK: %[[YDECL:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE]] arg 2
! CHECK: omp.declare_simd uniform(%[[XDECL]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, %[[YDECL]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>)
+! CHECK-SAME: {arg_types = [f64, f64, i32, i32]}
! CHECK: return
subroutine declare_simd_inbranch()
@@ -151,17 +155,17 @@ end subroutine declare_simd_combined
! CHECK: %[[N_DECL:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE]] arg 3 {{.*}} : (!fir.ref<i32>, !fir.dscope) -> (!fir.ref<i32>, !fir.ref<i32>)
! CHECK: %[[X_DECL:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE]] arg 1 {{.*}} : (!fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, !fir.dscope) -> (!fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>)
! CHECK: %[[Y_DECL:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE]] arg 2 {{.*}} : (!fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, !fir.dscope) -> (!fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>, !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>)
-! CHECK: %[[C1:.*]] = arith.constant 1 : i32
+! CHECK: %[[C4:.*]] = arith.constant 4 : i32
! CHECK: omp.declare_simd
! CHECK-SAME: aligned(%[[X_DECL]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>> -> 64 : i64,
! CHECK-SAME: %[[Y_DECL]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>> -> 64 : i64)
! CHECK-SAME: inbranch
-! CHECK-SAME: linear(ref(%[[I_DECL]]#0 : !fir.ref<i32> = %[[C1]] : i32))
+! CHECK-SAME: linear(ref(%[[I_DECL]]#0 : !fir.ref<i32> = %[[C4]] : i32))
! CHECK-SAME: simdlen(8)
! CHECK-SAME: uniform(%[[X_DECL]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>,
! CHECK-SAME: %[[Y_DECL]]#0 : !fir.ref<!fir.box<!fir.ptr<!fir.array<?xf64>>>>)
-! CHECK-SAME: {linear_var_types = [i32]}{{$}}
+! CHECK-SAME: {arg_types = [f64, f64, i32, i32], linear_var_types = [i32]}{{$}}
! CHECK: return
subroutine declare_simd_linear_val(a, b)
@@ -180,7 +184,7 @@ end subroutine declare_simd_linear_val
! CHECK: %[[C2:.*]] = arith.constant 2 : i32
! CHECK: %[[C1:.*]] = arith.constant 1 : i32
! CHECK: omp.declare_simd linear(val(%[[A]]#0 : !fir.ref<i32> = %[[C2]] : i32), val(%[[B]]#0 : !fir.ref<i32> = %[[C1]] : i32))
-! CHECK-SAME: {linear_var_types = [i32, i32]}{{$}}
+! CHECK-SAME: {arg_types = [i32, i32], linear_var_types = [i32, i32]}{{$}}
! CHECK: return
subroutine declare_simd_linear_ref(x)
@@ -195,9 +199,9 @@ end subroutine declare_simd_linear_ref
! CHECK-LABEL: func.func @_QPdeclare_simd_linear_ref(
! CHECK: %[[SCOPE:.*]] = fir.dummy_scope : !fir.dscope
! CHECK: %[[X:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE]] arg 1 {{.*}} : (!fir.ref<!fir.box<!fir.heap<i32>>>, !fir.dscope) -> (!fir.ref<!fir.box<!fir.heap<i32>>>, !fir.ref<!fir.box<!fir.heap<i32>>>)
-! CHECK: %[[C4:.*]] = arith.constant 4 : i32
-! CHECK: omp.declare_simd linear(ref(%[[X]]#0 : !fir.ref<!fir.box<!fir.heap<i32>>> = %[[C4]] : i32))
-! CHECK-SAME: {linear_var_types = [!fir.box<!fir.heap<i32>>]}{{$}}
+! CHECK: %[[C16:.*]] = arith.constant 16 : i32
+! CHECK: omp.declare_simd linear(ref(%[[X]]#0 : !fir.ref<!fir.box<!fir.heap<i32>>> = %[[C16]] : i32))
+! CHECK-SAME: {arg_types = [i32], linear_var_types = [!fir.box<!fir.heap<i32>>]}{{$}}
! CHECK: return
subroutine declare_simd_linear_uval(y)
@@ -214,5 +218,5 @@ end subroutine declare_simd_linear_uval
! CHECK: %[[Y:.*]]:2 = hlfir.declare %{{.*}} dummy_scope %[[SCOPE]] arg 1 {{.*}} : (!fir.ref<i32>, !fir.dscope) -> (!fir.ref<i32>, !fir.ref<i32>)
! CHECK: %[[C1:.*]] = arith.constant 1 : i32
! CHECK: omp.declare_simd linear(uval(%[[Y]]#0 : !fir.ref<i32> = %[[C1]] : i32))
-! CHECK-SAME: {linear_var_types = [i32]}{{$}}
+! CHECK-SAME: {arg_types = [i32], linear_var_types = [i32]}{{$}}
! CHECK: return
diff --git a/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td b/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
index 88c8ab4f6f949..dc5255aaab47e 100644
--- a/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
+++ b/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
@@ -2285,6 +2285,12 @@ def DeclareSimdOp
a function body. It attaches clauses of declare simd to the enclosing
function.
+ The optional `arg_types` attribute records the original language-level
+ types of the enclosing function's arguments. This is used during
+ translation to recover pointee-type information lost in opaque
+ `!llvm.ptr`, enabling correct lane-size (LS) computation for the
+ AArch64 AAVFABI.
+
Example:
```mlir
func.func @add(%a: memref<16xi32>) {
@@ -2294,6 +2300,8 @@ def DeclareSimdOp
```
}] # clausesDescription;
+ let arguments = !con(clausesArgs, (ins OptionalAttr<ArrayAttr>:$arg_types));
+
let builders = [OpBuilder<(
ins CArg<"const DeclareSimdOperands &">:$clauses)>];
diff --git a/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp b/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
index 04418ee39be54..ca79b12b68269 100644
--- a/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
+++ b/mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp
@@ -4707,6 +4707,13 @@ LogicalResult DeclareSimdOp::verify() {
if (getInbranch() && getNotinbranch())
return emitOpError("cannot have both 'inbranch' and 'notinbranch'");
+ if (auto argTypes = getArgTypes()) {
+ if (argTypes->size() != func.getNumArguments())
+ return emitOpError() << "'arg_types' length (" << argTypes->size()
+ << ") must match the number of function arguments ("
+ << func.getNumArguments() << ")";
+ }
+
if (failed(verifyLinearModifiers(*this, getLinearModifiers(), getLinearVars(),
/*isDeclareSimd=*/true)))
return failure();
@@ -4722,7 +4729,8 @@ void DeclareSimdOp::build(OpBuilder &odsBuilder, OperationState &odsState,
clauses.linearVars, clauses.linearStepVars,
clauses.linearVarTypes, clauses.linearModifiers,
clauses.notinbranch, clauses.simdlen,
- clauses.uniformVars);
+ clauses.uniformVars,
+ /*arg_types=*/nullptr);
}
//===----------------------------------------------------------------------===//
diff --git a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
index 37b1a37c2e1a5..fb0f6636f8e8f 100644
--- a/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/Dialect/OpenMP/OpenMPToLLVMIRTranslation.cpp
@@ -7465,6 +7465,394 @@ convertTargetFreeMemOp(Operation &opInst, llvm::IRBuilderBase &builder,
return success();
}
+static void populateLinearParam(
+ llvm::DenseMap<Value, unsigned> &argIndexMap, omp::DeclareSimdOp ds,
+ llvm::SmallVectorImpl<llvm::OpenMPIRBuilder::DeclareSimdAttrTy> &attrs) {
+ OperandRange linearVars = ds.getLinearVars();
+ OperandRange linearStepVars = ds.getLinearStepVars();
+ std::optional<ArrayAttr> linearModifiers = ds.getLinearModifiers();
+
+ const llvm::APSInt defaultStep(llvm::APInt(/*numBits=*/32, /*val=*/1),
+ /*isUnsigned=*/true);
+
+ auto resolveStepArgIndex = [&](Value stepValue) -> std::optional<unsigned> {
+ // Var-stride can be expressed as `llvm.load %argN`.
+ if (auto load = stepValue.getDefiningOp<LLVM::LoadOp>())
+ stepValue = load.getAddr();
+
+ if (auto it = argIndexMap.find(stepValue); it != argIndexMap.end())
+ return it->second;
+
+ return std::nullopt;
+ };
+
+ for (size_t i = 0; i < linearVars.size(); ++i) {
+ llvm::OpenMPIRBuilder::DeclareSimdAttrTy ¶mAttr =
+ attrs[argIndexMap[linearVars[i]]];
+ omp::LinearModifierAttr linearModAttr;
+
+ if (linearModifiers && (*linearModifiers)[i])
+ linearModAttr = dyn_cast<omp::LinearModifierAttr>((*linearModifiers)[i]);
+
+ if (linearModAttr) {
+ switch (linearModAttr.getValue()) {
+ case omp::LinearModifier::ref:
+ paramAttr.Kind = llvm::OpenMPIRBuilder::DeclareSimdKindTy::LinearRef;
+ break;
+ case omp::LinearModifier::uval:
+ paramAttr.Kind = llvm::OpenMPIRBuilder::DeclareSimdKindTy::LinearUVal;
+ break;
+ case omp::LinearModifier::val:
+ // Match clang: val on a non-reference (non-pointer SSA type) is
+ // semantically identical to plain linear. Only pointer-typed
+ // vars (which may originate from C++ references) get LinearVal.
+ if (isa<LLVM::LLVMPointerType>(linearVars[i].getType()))
+ paramAttr.Kind = llvm::OpenMPIRBuilder::DeclareSimdKindTy::LinearVal;
+ else
+ paramAttr.Kind = llvm::OpenMPIRBuilder::DeclareSimdKindTy::Linear;
+ break;
+ }
+ } else {
+ paramAttr.Kind = llvm::OpenMPIRBuilder::DeclareSimdKindTy::Linear;
+ }
+
+ paramAttr.HasVarStride = false;
+ paramAttr.StrideOrArg = defaultStep;
+
+ if (i >= linearStepVars.size())
+ continue;
+
+ Value stepValue = linearStepVars[i];
+
+ if (std::optional<unsigned> stepArgIdx = resolveStepArgIndex(stepValue)) {
+ paramAttr.HasVarStride = true;
+ paramAttr.StrideOrArg =
+ llvm::APSInt(llvm::APInt(/*numBits=*/32, *stepArgIdx),
+ /*isUnsigned=*/true);
+ continue;
+ }
+
+ if (auto cst = stepValue.getDefiningOp<LLVM::ConstantOp>()) {
+ IntegerAttr intAttr = cast<IntegerAttr>(cst.getValue());
+ paramAttr.HasVarStride = false;
+ paramAttr.StrideOrArg =
+ llvm::APSInt(intAttr.getValue(), /*isUnsigned=*/false);
+ continue;
+ }
+
+ llvm_unreachable("unhandled linear step form");
+ }
+}
+
+static llvm::OpenMPIRBuilder::DeclareSimdBranch
+getDeclareSimdBranch(omp::DeclareSimdOp &op) {
+ if (op.getInbranch())
+ return llvm::OpenMPIRBuilder::DeclareSimdBranch::Inbranch;
+ if (op.getNotinbranch())
+ return llvm::OpenMPIRBuilder::DeclareSimdBranch::Notinbranch;
+ return llvm::OpenMPIRBuilder::DeclareSimdBranch::Undefined;
+}
+
+static unsigned
+evaluateCDTSize(const llvm::Function *fn,
+ ArrayRef<llvm::OpenMPIRBuilder::DeclareSimdAttrTy> paramAttrs) {
+ // Every vector variant of a SIMD-enabled function has a vector length (VLEN).
+ // If OpenMP clause "simdlen" is used, the VLEN is the value of the argument
+ // of that clause. The VLEN value must be power of 2.
+ // In other case the notion of the function`s "characteristic data type" (CDT)
+ // is used to compute the vector length.
+ // CDT is defined in the following order:
+ // a) For non-void function, the CDT is the return type.
+ // b) If the function has any non-uniform, non-linear parameters, then the
+ // CDT is the type of the first such parameter.
+ // c) If the CDT determined by a) or b) above is struct, union, or class
+ // type which is pass-by-value (except for the type that maps to the
+ // built-in complex data type), the characteristic data type is int.
+ // d) If none of the above three cases is applicable, the CDT is int.
+ // The VLEN is then determined based on the CDT and the size of vector
+ // register of that ISA for which current vector version is generated. The
+ // VLEN is computed using the formula below:
+ // VLEN = sizeof(vector_register) / sizeof(CDT),
+ // where vector register size specified in section 3.2.1 Registers and the
+ // Stack Frame of original AMD64 ABI document.
+ const llvm::DataLayout &dl = fn->getParent()->getDataLayout();
+ llvm::Type *cdtTy = nullptr;
+
+ // For a non-void function, the return type is the characteristic data type.
+ llvm::Type *retTy = fn->getReturnType();
+ if (retTy && !retTy->isVoidTy())
+ cdtTy = retTy;
+
+ // Otherwise, use the first parameter that is still vectorized. Parameters
+ // without an explicit declare simd attribute are treated as vector
+ // parameters because Vector is the default ki...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/187767
More information about the llvm-branch-commits
mailing list