[clang] [clang] Reject VLAs in `__is_layout_compatible()` (PR #87737)

[via cfe-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-clang

Author: Vlad Serebrennikov (Endilll)

<details>
<summary>Changes</summary>

This is a follow-up to #<!-- -->81506. Since `__is_layout_compatible()` is a C++ intrinsic (https://github.com/llvm/llvm-project/blob/ff1e72d68d1224271801ff5192a8c14fbd3be83b/clang/include/clang/Basic/TokenKinds.def#L523), I don't think we should define how it interacts with VLA extension unless we have a compelling reason to.

Since #<!-- -->81506 was merged after 18 cut-off, we don't have to follow any kind of deprecation process.

---
Full diff: https://github.com/llvm/llvm-project/pull/87737.diff


2 Files Affected:

- (modified) clang/lib/Sema/SemaExprCXX.cpp (+3) 
- (modified) clang/test/SemaCXX/type-traits.cpp (+4-2) 


``````````diff
diff --git a/clang/lib/Sema/SemaExprCXX.cpp b/clang/lib/Sema/SemaExprCXX.cpp
index 76bb78aa8b5458..db84f181012268 100644
--- a/clang/lib/Sema/SemaExprCXX.cpp
+++ b/clang/lib/Sema/SemaExprCXX.cpp
@@ -6026,6 +6026,9 @@ static bool EvaluateBinaryTypeTrait(Sema &Self, TypeTrait BTT, QualType LhsT,
     return false;
   }
   case BTT_IsLayoutCompatible: {
+    if (LhsT->isVariableArrayType() || RhsT->isVariableArrayType())
+      Self.Diag(KeyLoc, diag::err_vla_unsupported)
+          << 1 << tok::kw___is_layout_compatible;
     return Self.IsLayoutCompatible(LhsT, RhsT);
   }
     default: llvm_unreachable("not a BTT");
diff --git a/clang/test/SemaCXX/type-traits.cpp b/clang/test/SemaCXX/type-traits.cpp
index 14ec17989ec7c7..28653e82e5a16e 100644
--- a/clang/test/SemaCXX/type-traits.cpp
+++ b/clang/test/SemaCXX/type-traits.cpp
@@ -1741,8 +1741,10 @@ void is_layout_compatible(int n)
   static_assert(!__is_layout_compatible(unsigned char, signed char));
   static_assert(__is_layout_compatible(int[], int[]));
   static_assert(__is_layout_compatible(int[2], int[2]));
-  static_assert(!__is_layout_compatible(int[n], int[2])); // FIXME: VLAs should be rejected
-  static_assert(!__is_layout_compatible(int[n], int[n])); // FIXME: VLAs should be rejected
+  static_assert(!__is_layout_compatible(int[n], int[2]));
+  // expected-error at -1 {{variable length arrays are not supported for '__is_layout_compatible'}}
+  static_assert(!__is_layout_compatible(int[n], int[n]));
+  // expected-error at -1 {{variable length arrays are not supported for '__is_layout_compatible'}}
   static_assert(__is_layout_compatible(int&, int&));
   static_assert(!__is_layout_compatible(int&, char&));
   static_assert(__is_layout_compatible(void(int), void(int)));

``````````

</details>


https://github.com/llvm/llvm-project/pull/87737