[flang-commits] [flang] [flang][runtime] Fix integer overflow check for FORMATs (PR #79471)

[Peter Klausler via flang-commits]

https://github.com/klausler created https://github.com/llvm/llvm-project/pull/79471

The code that parses repeat counts, field widths, &c. from FORMAT strings has an incorrect overflow check, so the maximum integer value is not accepted.  Fix.

Fixes https://github.com/llvm/llvm-project/issues/79255.

>From 118dd707d18e4cb581030e54c754e57d8a6dbd32 Mon Sep 17 00:00:00 2001
From: Peter Klausler <pklausler at nvidia.com>
Date: Thu, 25 Jan 2024 09:00:09 -0800
Subject: [PATCH] [flang][runtime] Fix integer overflow check for FORMATs

The code that parses repeat counts, field widths, &c. from FORMAT
strings has an incorrect overflow check, so the maximum integer
value is not accepted.  Fix.

Fixes https://github.com/llvm/llvm-project/issues/79255.
---
 flang/runtime/format-implementation.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/flang/runtime/format-implementation.h b/flang/runtime/format-implementation.h
index c54ac062c7beab..b0a4f6b9eb2f79 100644
--- a/flang/runtime/format-implementation.h
+++ b/flang/runtime/format-implementation.h
@@ -85,8 +85,8 @@ int FormatControl<CONTEXT>::GetIntField(
     ch = PeekNext();
   }
   while (ch >= '0' && ch <= '9') {
-    if (result >
-        std::numeric_limits<int>::max() / 10 - (static_cast<int>(ch) - '0')) {
+    constexpr int tenth{std::numeric_limits<int>::max() / 10};
+    if (result > tenth || ch - '0' > std::numeric_limits<int>::max() - result) {
       handler.SignalError(
           IostatErrorInFormat, "FORMAT integer field out of range");
       if (hadError) {