[flang-commits] [flang] [flang][runtime] Fix integer overflow check in FORMAT (PR #79368)

[via flang-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-flang-runtime

Author: Peter Klausler (klausler)

<details>
<summary>Changes</summary>

The integer overflow check used for repeat counts &c. in FORMAT specifications was incorrect; fix it.

Fixes https://github.com/llvm/llvm-project/issues/79255.

---
Full diff: https://github.com/llvm/llvm-project/pull/79368.diff


1 Files Affected:

- (modified) flang/runtime/format-implementation.h (+3-2) 


``````````diff
diff --git a/flang/runtime/format-implementation.h b/flang/runtime/format-implementation.h
index c54ac062c7beab9..e475508b9ba58a7 100644
--- a/flang/runtime/format-implementation.h
+++ b/flang/runtime/format-implementation.h
@@ -85,8 +85,9 @@ int FormatControl<CONTEXT>::GetIntField(
     ch = PeekNext();
   }
   while (ch >= '0' && ch <= '9') {
-    if (result >
-        std::numeric_limits<int>::max() / 10 - (static_cast<int>(ch) - '0')) {
+    if (result > std::numeric_limits<int>::max() / 10 ||
+        std::numeric_limits<int>::max() - 10 * result <
+            static_cast<int>(ch) - '0') {
       handler.SignalError(
           IostatErrorInFormat, "FORMAT integer field out of range");
       if (hadError) {

``````````

</details>


https://github.com/llvm/llvm-project/pull/79368