[flang-commits] [flang] [flang][docs] add design doc for inline runtime checks (PR #228502)

[Eugene Epshteyn via flang-commits]

================
@@ -0,0 +1,785 @@
+<!--===- docs/InlineRuntimeChecks.md
+
+   Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+   See https://llvm.org/LICENSE.txt for license information.
+   SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+-->
+
+# Inline runtime checks in Flang with `fir.assert`
+
+Status: draft design, for discussion.
+Context: [llvm/llvm-project#223287](https://github.com/llvm/llvm-project/pull/223287).
+Source references are to `llvm-project` at `fe1de64cb3c2` (2026-10-02). Line
+numbers drift, so each reference also names the function or pattern.
+
+Abbreviations used in the tables:
+
+| Short | Path |
+|---|---|
+| `SHI` | `flang/lib/Optimizer/HLFIR/Transforms/SimplifyHLFIRIntrinsics.cpp` |
+| `OB` | `flang/lib/Optimizer/HLFIR/Transforms/OptimizedBufferization.cpp` |
+| `IHA` | `flang/lib/Optimizer/HLFIR/Transforms/InlineHLFIRAssign.cpp` |
+| `SAA` | `flang/lib/Optimizer/HLFIR/Transforms/SeparateAllocatableAssign.cpp` |
+| `C2F` | `flang/lib/Optimizer/HLFIR/Transforms/ConvertToFIR.cpp` |
+| `SI` | `flang/lib/Optimizer/Transforms/SimplifyIntrinsics.cpp` (FIR level) |
+| `IC` | `flang/lib/Optimizer/Builder/IntrinsicCall.cpp` |
+| `X2H` | `flang/lib/Lower/ConvertExprToHLFIR.cpp` |
+| `RT/` | `flang-rt/lib/runtime/` |
+
+Terminology:
+
+- **Lowering** is the translation of the Fortran parse tree to HLFIR, in
+  `flang/lib/Lower`.
+- **Emitting** an assert means creating a `fir.assert` operation, either in
+  lowering or in an HLFIR or FIR pass.
+- **Code generation** is the conversion of FIR to the LLVM dialect
+  (`FIRToLLVMLowering` in `flang/lib/Optimizer/CodeGen/CodeGen.cpp`), where
+  `fir.assert` becomes a branch and a runtime call.
+
+## 1. Problem
+
+Flang relies on the Fortran runtime for almost all dynamic checks: shape
+conformance, allocation status, argument consistency of intrinsics. Every
+time an HLFIR or FIR pass replaces a runtime call with inline code, the check
+silently disappears. Three consequences:
+
+1. **Behavior depends on the optimization level.** The inlining passes mostly
+   run at `-O1` and above, so the same non-conforming program stops with a
+   clear message at `-O0` and does something else at `-O2`.
+2. **The inline code is often memory-unsafe on non-conforming input, not just
+   wrong.** Loop bounds are taken from one operand and used to index another
+   (`fir::factory::deduceOptimalExtents`). A size mismatch becomes an
+   out-of-bounds read or write instead of an error.
+3. **Some checks the standard requires are missing on every path.** For
+   example, the inline ALLOCATE/DEALLOCATE of intrinsic-type allocatables
+   does not check the allocation status (F2018 9.7.1.3, 9.7.3.2, 9.7.4).
+
+The few inline checks that exist today (MOD/MODULO `P==0`, NEAREST `S==0`,
+STORAGE_SIZE, the realloc-with-scalar-RHS case) use `fir.if` plus
+`fir.call @_FortranAReportFatalUserError`. That pattern:
+
+- has no data dependence on the operation it protects, so passes may move the
+  protected operation above the check (MLIR models neither "may not return"
+  nor non-local control flow: `mlir/docs/Rationale/SideEffectsAndSpeculation.md`);
+- introduces a call with unknown memory effects, which blocks alias-based
+  optimizations;
+- does not tell LLVM the call never returns (the FIR declaration has no
+  `noreturn`), so LLVM cannot use the checked condition afterwards.
+
+## 2. Goals and non-goals
+
+Goals:
+
+- Keep every user-facing check the runtime performs when the call is inlined,
+  and report it with the runtime's message format,
+  `fatal Fortran runtime error(<file>:<line>): <message>`, and with message
+  text as close as possible to the runtime's.
+- Make non-conforming programs behave the same at `-O0` and `-O2`, so that
+  tests disabled because of that difference (for example in the
+  llvm-test-suite gfortran tests) can be re-enabled.
+- Keep the checks cheap and transparent to optimization. They must not block
+  store-to-load forwarding, CSE of loads, LICM of loads, or alias analysis.
+- Let checks be hoisted, merged, folded, and dropped under an option.
+- Provide one mechanism for future opt-in checks (`-fcheck=bounds`,
+  `pointer`, `do`, `bits`).
+
+Non-goals:
+
+- Detecting undefined POINTER association status (only disassociated, that
+  is null, is detectable).
+- Checking explicit-shape dummy size against the actual argument in general
+  (the callee only receives an address).
+- Changing the runtime's own checks, apart from the bugs listed in the
+  appendix.
+
+## 3. The `fir.assert` operation
+
+### 3.1 Semantics and syntax
+
+```
+%g:N = fir.assert %ok, <kind>, "<message>" [values(%v0, ... : T0, ...)] [guard(%x0, ... : U0, ...)]
+```
+
+- `%ok` (`i1`) is the condition that must hold.
+- `<kind>` is a check category (section 4). Options act on categories.
+- `<message>` is the error text, a `printf` format.
+- `values(%v0, ... : T0, ...)` are optional integer or `index` SSA values
+  (`T0`, ... are their types) printed into the message, one per `%jd`
+  directive. They are only read on the failure path. For example, the two
+  extents in `"DOT_PRODUCT: SIZE(VECTOR_A) is %jd but SIZE(VECTOR_B) is %jd"`.
+- `guard(%x0, ... : U0, ...)` are the optional guarded values, of any type.
+  The op has one result per guarded value, with the same type (`%g#i` has
+  type `Ui`).
+- If `%ok` is true, each result equals the corresponding guarded value and
+  nothing else happens. Otherwise the program terminates through the Fortran
+  runtime with the formatted message.
+- Operations that must not run before the check consume the guarded results.
+  The ordering is then enforced by SSA dominance, for every pass, without any
+  pass having to know about `fir.assert`. Asserts without a guard only order
+  themselves with respect to I/O and calls (section 3.2).
+
+What to guard: the operand of the protected operation, such as a divisor,
+a DO step, a subscript, or a box that is dereferenced. For conformance
+checks, it is the **array being accessed** (a variable, or an
+`!hlfir.expr`), at the point where it is combined with another array:
+
+- In an elemental operation or an elemental procedure reference, guard the
+  array operands that do not provide the iteration space.
+- In an array assignment, guard both sides. The pass that expands the
+  assignment may iterate over either one.
+
+Conformance checks do not guard shapes or extents. HLFIR can re-derive them
+without going
+through a guarded value: from static types (`hlfir.shape_of` folds to a
+constant shape), from the preference for constant extents
+(`fir::factory::deduceOptimalExtents`), or from the other side of an
+assignment. Leaving them unguarded also keeps constant loop bounds visible
+to MLIR loop optimizations. Array bases are never compile-time constants, so
+guarding them loses nothing. Section 7 explains how the guards follow an
+expression when it is inlined.
+
+### 3.2 Traits
+
+- `MemoryEffects<[MemWrite<FortranRuntimeResource>]>`, where
+  `FortranRuntimeResource` is a non-addressable resource (like the existing
+  `fir::DebuggingResource`).
+  - The op is never trivially dead, so DCE keeps asserts without uses.
+  - It stays ordered with calls and I/O, whose unknown effects cover every
+    resource.
+  - It does not touch user memory. FIR alias analysis skips non-addressable
+    resources (`flang/lib/Optimizer/Analysis/AliasAnalysis.cpp`, `getModRef`),
+    and MLIR CSE ignores writes to resources disjoint from a load's resource.
+    So asserts do not block load CSE, store-to-load forwarding, or FIR LICM of
+    loads. This also means `SimplifyHLFIRIntrinsics` can emit asserts in its
+    first run (`allowNewSideEffects=false`, `SHI:3321-3333`): the concern
+    there is new effects that stop CSE from merging loads, which this op does
+    not have.
+- Not speculatable.
+- **Not transparent to other operations' folds and patterns.** The assert's
+  own canonicalization removes it when `%ok` is a constant true (section 3.4).
+  But no other rewrite may look through an assert to its guarded operand,
+  because that drops the dependence. For example, in
+  `%a = fir.box_addr %b; %a_ok = fir.assert ... guard(%a); %c = fir.embox %a_ok`,
+  a fold that rewrites an `fir.embox` of a `fir.box_addr` into the original
+  box must not replace `%c` with `%b`. In practice:
+  - the op does not implement `ViewLikeOpInterface`;
+  - it is not added to the helpers that skip `fir.declare` or `fir.convert`
+    to find a defining op;
+  - it is not given a folder that returns a guarded operand when `%ok` is
+    unknown.
+
+  Alias analysis may look through the assert, since it only answers
----------------
eugeneepshteyn wrote:

With A1 guarding both sides of every array assignment, "may look through" needs to become a §8 step-1 prerequisite. Everything that finds its subject through the defining op degrades through a guard: `fir::AliasAnalysis` sources (→ MayAlias) and so `AddAliasTags`, `ArraySectionAnalyzer`, OptimizedBufferization's `findMatch` / `tryUsingAssignLhsDirectly`, `ScheduleOrderedAssignments` (a value-less write is an "any" conflict), `isSimplyContiguous`, `mayBeOptional`, `genShape`/`ShapeOfOp::fold`. Substituting an opaque pass-through op for both sides of a plain `x = y` between assumed-shape dummies makes `--inline-hlfir-assign` emit a runtime overlap test, a heap temporary and three loops instead of one. Shape/length queries can look through `conformance` and `argument` guards, not `pointer` ones. Relatedly, line 138: array bases carry provenance (global, alloca, dummy) that a guard hides, so guarding them is not free until AA looks through. Guarded `!hlfir.expr` values also need an explicit BufferizeHLFIR pattern (full conversion) and the `hlfir.no_reassoc`-like treatment on the associate buffer-reuse path.


https://github.com/llvm/llvm-project/pull/228502