[flang-commits] [flang] [flang][docs] add design doc for inline runtime checks (PR #228502)

[via flang-commits]

================
@@ -0,0 +1,785 @@
+<!--===- docs/InlineRuntimeChecks.md
+
+   Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+   See https://llvm.org/LICENSE.txt for license information.
+   SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+-->
+
+# Inline runtime checks in Flang with `fir.assert`
+
+Status: draft design, for discussion.
+Context: [llvm/llvm-project#223287](https://github.com/llvm/llvm-project/pull/223287).
+Source references are to `llvm-project` at `fe1de64cb3c2` (2026-10-02). Line
+numbers drift, so each reference also names the function or pattern.
+
+Abbreviations used in the tables:
+
+| Short | Path |
+|---|---|
+| `SHI` | `flang/lib/Optimizer/HLFIR/Transforms/SimplifyHLFIRIntrinsics.cpp` |
+| `OB` | `flang/lib/Optimizer/HLFIR/Transforms/OptimizedBufferization.cpp` |
+| `IHA` | `flang/lib/Optimizer/HLFIR/Transforms/InlineHLFIRAssign.cpp` |
+| `SAA` | `flang/lib/Optimizer/HLFIR/Transforms/SeparateAllocatableAssign.cpp` |
+| `C2F` | `flang/lib/Optimizer/HLFIR/Transforms/ConvertToFIR.cpp` |
+| `SI` | `flang/lib/Optimizer/Transforms/SimplifyIntrinsics.cpp` (FIR level) |
+| `IC` | `flang/lib/Optimizer/Builder/IntrinsicCall.cpp` |
+| `X2H` | `flang/lib/Lower/ConvertExprToHLFIR.cpp` |
+| `RT/` | `flang-rt/lib/runtime/` |
+
+Terminology:
+
+- **Lowering** is the translation of the Fortran parse tree to HLFIR, in
+  `flang/lib/Lower`.
+- **Emitting** an assert means creating a `fir.assert` operation, either in
+  lowering or in an HLFIR or FIR pass.
+- **Code generation** is the conversion of FIR to the LLVM dialect
+  (`FIRToLLVMLowering` in `flang/lib/Optimizer/CodeGen/CodeGen.cpp`), where
+  `fir.assert` becomes a branch and a runtime call.
+
+## 1. Problem
+
+Flang relies on the Fortran runtime for almost all dynamic checks: shape
+conformance, allocation status, argument consistency of intrinsics. Every
+time an HLFIR or FIR pass replaces a runtime call with inline code, the check
+silently disappears. Three consequences:
+
+1. **Behavior depends on the optimization level.** The inlining passes mostly
+   run at `-O1` and above, so the same non-conforming program stops with a
+   clear message at `-O0` and does something else at `-O2`.
+2. **The inline code is often memory-unsafe on non-conforming input, not just
+   wrong.** Loop bounds are taken from one operand and used to index another
+   (`fir::factory::deduceOptimalExtents`). A size mismatch becomes an
+   out-of-bounds read or write instead of an error.
+3. **Some checks the standard requires are missing on every path.** For
+   example, the inline ALLOCATE/DEALLOCATE of intrinsic-type allocatables
+   does not check the allocation status (F2018 9.7.1.3, 9.7.3.2, 9.7.4).
+
+The few inline checks that exist today (MOD/MODULO `P==0`, NEAREST `S==0`,
+STORAGE_SIZE, the realloc-with-scalar-RHS case) use `fir.if` plus
+`fir.call @_FortranAReportFatalUserError`. That pattern:
+
+- has no data dependence on the operation it protects, so passes may move the
+  protected operation above the check (MLIR models neither "may not return"
+  nor non-local control flow: `mlir/docs/Rationale/SideEffectsAndSpeculation.md`);
+- introduces a call with unknown memory effects, which blocks alias-based
+  optimizations;
+- does not tell LLVM the call never returns (the FIR declaration has no
+  `noreturn`), so LLVM cannot use the checked condition afterwards.
+
+## 2. Goals and non-goals
+
+Goals:
+
+- Keep every user-facing check the runtime performs when the call is inlined,
+  and report it with the runtime's message format,
+  `fatal Fortran runtime error(<file>:<line>): <message>`, and with message
+  text as close as possible to the runtime's.
+- Make non-conforming programs behave the same at `-O0` and `-O2`, so that
+  tests disabled because of that difference (for example in the
+  llvm-test-suite gfortran tests) can be re-enabled.
+- Keep the checks cheap and transparent to optimization. They must not block
+  store-to-load forwarding, CSE of loads, LICM of loads, or alias analysis.
+- Let checks be hoisted, merged, folded, and dropped under an option.
+- Provide one mechanism for future opt-in checks (`-fcheck=bounds`,
+  `pointer`, `do`, `bits`).
+
+Non-goals:
+
+- Detecting undefined POINTER association status (only disassociated, that
+  is null, is detectable).
+- Checking explicit-shape dummy size against the actual argument in general
+  (the callee only receives an address).
+- Changing the runtime's own checks, apart from the bugs listed in the
+  appendix.
+
+## 3. The `fir.assert` operation
+
+### 3.1 Semantics and syntax
+
+```
+%g:N = fir.assert %ok, <kind>, "<message>" [values(%v0, ... : T0, ...)] [guard(%x0, ... : U0, ...)]
+```
+
+- `%ok` (`i1`) is the condition that must hold.
+- `<kind>` is a check category (section 4). Options act on categories.
+- `<message>` is the error text, a `printf` format.
+- `values(%v0, ... : T0, ...)` are optional integer or `index` SSA values
+  (`T0`, ... are their types) printed into the message, one per `%jd`
+  directive. They are only read on the failure path. For example, the two
+  extents in `"DOT_PRODUCT: SIZE(VECTOR_A) is %jd but SIZE(VECTOR_B) is %jd"`.
+- `guard(%x0, ... : U0, ...)` are the optional guarded values, of any type.
+  The op has one result per guarded value, with the same type (`%g#i` has
+  type `Ui`).
+- If `%ok` is true, each result equals the corresponding guarded value and
+  nothing else happens. Otherwise the program terminates through the Fortran
+  runtime with the formatted message.
+- Operations that must not run before the check consume the guarded results.
+  The ordering is then enforced by SSA dominance, for every pass, without any
+  pass having to know about `fir.assert`. Asserts without a guard only order
+  themselves with respect to I/O and calls (section 3.2).
+
+What to guard: the operand of the protected operation, such as a divisor,
+a DO step, a subscript, or a box that is dereferenced. For conformance
+checks, it is the **array being accessed** (a variable, or an
+`!hlfir.expr`), at the point where it is combined with another array:
+
+- In an elemental operation or an elemental procedure reference, guard the
+  array operands that do not provide the iteration space.
+- In an array assignment, guard both sides. The pass that expands the
+  assignment may iterate over either one.
+
+Conformance checks do not guard shapes or extents. HLFIR can re-derive them
+without going
+through a guarded value: from static types (`hlfir.shape_of` folds to a
+constant shape), from the preference for constant extents
+(`fir::factory::deduceOptimalExtents`), or from the other side of an
+assignment. Leaving them unguarded also keeps constant loop bounds visible
+to MLIR loop optimizations. Array bases are never compile-time constants, so
+guarding them loses nothing. Section 7 explains how the guards follow an
+expression when it is inlined.
+
+### 3.2 Traits
+
+- `MemoryEffects<[MemWrite<FortranRuntimeResource>]>`, where
+  `FortranRuntimeResource` is a non-addressable resource (like the existing
+  `fir::DebuggingResource`).
+  - The op is never trivially dead, so DCE keeps asserts without uses.
+  - It stays ordered with calls and I/O, whose unknown effects cover every
+    resource.
+  - It does not touch user memory. FIR alias analysis skips non-addressable
+    resources (`flang/lib/Optimizer/Analysis/AliasAnalysis.cpp`, `getModRef`),
+    and MLIR CSE ignores writes to resources disjoint from a load's resource.
+    So asserts do not block load CSE, store-to-load forwarding, or FIR LICM of
+    loads. This also means `SimplifyHLFIRIntrinsics` can emit asserts in its
+    first run (`allowNewSideEffects=false`, `SHI:3321-3333`): the concern
+    there is new effects that stop CSE from merging loads, which this op does
+    not have.
+- Not speculatable.
+- **Not transparent to other operations' folds and patterns.** The assert's
+  own canonicalization removes it when `%ok` is a constant true (section 3.4).
+  But no other rewrite may look through an assert to its guarded operand,
+  because that drops the dependence. For example, in
+  `%a = fir.box_addr %b; %a_ok = fir.assert ... guard(%a); %c = fir.embox %a_ok`,
+  a fold that rewrites an `fir.embox` of a `fir.box_addr` into the original
+  box must not replace `%c` with `%b`. In practice:
+  - the op does not implement `ViewLikeOpInterface`;
+  - it is not added to the helpers that skip `fir.declare` or `fir.convert`
+    to find a defining op;
+  - it is not given a folder that returns a guarded operand when `%ok` is
+    unknown.
+
+  Alias analysis may look through the assert, since it only answers
----------------
jeanPerier wrote:

Agreed, this is a real cost. Section 3.2 now says analyses must look through guards. That means alias analysis, and the helpers that find a variable's origin, attributes, contiguity or optionality through its defining operation. Otherwise guarding both sides of assignments would pessimize them, so this is now a prerequisite in step 1 of section 8. The doc also records that shape and length queries may look through conformance and argument guards, but not pointer ones. The "guarding loses nothing" sentence now mentions the hidden provenance.

ScheduleOrderedAssignments is now listed next to InlineElementals as a pass that must skip non-addressable effects. For guarded !hlfir.expr values, the doc says BufferizeHLFIR moves the guard to the buffer, as it does for hlfir.no_reassoc. I'd rather leave the per-helper list to the implementation reviews.

https://github.com/llvm/llvm-project/pull/228502