[clang] [clang][Sema] Fix for enums overflowing (#24667) (PR #78742)

[via cfe-commits]

https://github.com/wheatman created https://github.com/llvm/llvm-project/pull/78742

Enums which do not have a specified type can only grow to bigger types which contain more bits than the prior types.  This means that the largest signed integer type cannot grow to the largest unsigned integer types.

Fixes #24667

>From 0bee966adc8c16ecdf6aa6d8a47f4837ebc3d771 Mon Sep 17 00:00:00 2001
From: Brian Wheatman <bwheatman at gmail.com>
Date: Fri, 19 Jan 2024 11:13:33 -0500
Subject: [PATCH] [clang][Sema] Fix for enums overflowing (#24667)

Enums which do not have a specified type can only grow to bigger types
which contain more bits than the prior types.  This means that the
largest signed integer type cannot grow to the largest unsigned integer types.

Fixes #24667
---
 clang/docs/ReleaseNotes.rst |  2 ++
 clang/lib/Sema/SemaDecl.cpp | 43 ++++++++++++++++++++++++++++++-------
 clang/test/SemaCXX/enum.cpp |  2 ++
 3 files changed, 39 insertions(+), 8 deletions(-)

diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 392f694065a242..bb6d8f256bbb07 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -731,6 +731,8 @@ Bug Fixes in This Version
   label (including ``case`` or ``default`` labels).
 - Fix compiler memory leak for enums with underlying type larger than 64 bits.
   Fixes (`#78311 <https://github.com/llvm/llvm-project/pull/78311>`_)
+- Fixes miscompilation when an enum has a specified value such that the auto 
+  increment overflows a signed long. Fixes (`#24667 <https://github.com/llvm/llvm-project/issues/24667>`_)
 
   Before:
 
diff --git a/clang/lib/Sema/SemaDecl.cpp b/clang/lib/Sema/SemaDecl.cpp
index cbb8ed8ce34d1b..fd63b7296c52b9 100644
--- a/clang/lib/Sema/SemaDecl.cpp
+++ b/clang/lib/Sema/SemaDecl.cpp
@@ -19768,20 +19768,47 @@ EnumConstantDecl *Sema::CheckEnumConstant(EnumDecl *Enum,
         //       sufficient to contain the incremented value. If no such type
         //       exists, the program is ill-formed.
         QualType T = getNextLargerIntegralType(Context, EltTy);
-        if (T.isNull() || Enum->isFixed()) {
+        if (Enum->isFixed()) {
           // There is no integral type larger enough to represent this
           // value. Complain, then allow the value to wrap around.
           EnumVal = LastEnumConst->getInitVal();
           EnumVal = EnumVal.zext(EnumVal.getBitWidth() * 2);
           ++EnumVal;
-          if (Enum->isFixed())
-            // When the underlying type is fixed, this is ill-formed.
-            Diag(IdLoc, diag::err_enumerator_wrapped)
-              << toString(EnumVal, 10)
-              << EltTy;
-          else
+          // When the underlying type is fixed, this is ill-formed.
+          Diag(IdLoc, diag::err_enumerator_wrapped)
+              << toString(EnumVal, 10) << EltTy;
+
+        } else if (T.isNull()) {
+          if (EltTy->isSignedIntegerType() &&
+              (getLangOpts().CPlusPlus ||
+               LangStandard::getLangStandardForKind(getLangOpts().LangStd)
+                   .isC23())) {
+            // FIXME: Int128/UInt128 support, which also needs to be introduced
+            // into
+            // enum checking below.
+            // This case can only happen if we are the largest signed type, so
+            // try and become an unsigned type to get more possible values
+            if (EltTy == Context.ShortTy) {
+              EltTy = Context.UnsignedShortTy;
+            } else if (EltTy == Context.IntTy) {
+              EltTy = Context.UnsignedIntTy;
+            } else if (EltTy == Context.LongTy) {
+              EltTy = Context.UnsignedLongTy;
+            } else if (EltTy == Context.LongLongTy) {
+              EltTy = Context.UnsignedLongLongTy;
+            } else {
+              assert(false && "Enum Type is not basic integral type");
+            }
+            
+          } else {
+            // There is no integral type larger enough to represent this
+            // value. Complain, then allow the value to wrap around.
+            EnumVal = LastEnumConst->getInitVal();
+            EnumVal = EnumVal.zext(EnumVal.getBitWidth() * 2);
+            ++EnumVal;
             Diag(IdLoc, diag::ext_enumerator_increment_too_large)
-              << toString(EnumVal, 10);
+                << toString(EnumVal, 10);
+          }
         } else {
           EltTy = T;
         }
diff --git a/clang/test/SemaCXX/enum.cpp b/clang/test/SemaCXX/enum.cpp
index fc65fd16f8c302..bee1d996f34fe8 100644
--- a/clang/test/SemaCXX/enum.cpp
+++ b/clang/test/SemaCXX/enum.cpp
@@ -125,3 +125,5 @@ struct PR28903 {
     })
   };
 };
+
+enum GH24667 { GH24667_x = 9223372036854775807 , GH24667_y };