[clang] 45ab2b4 - [Clang] Improve the handling of large arrays evaluation.
Corentin Jabot via cfe-commits
cfe-commits at lists.llvm.org
Fri Jul 28 05:38:27 PDT 2023
Author: Corentin Jabot
Date: 2023-07-28T14:38:22+02:00
New Revision: 45ab2b48bd55e50a86f6026ed31d2b60a118bdce
URL: https://github.com/llvm/llvm-project/commit/45ab2b48bd55e50a86f6026ed31d2b60a118bdce
DIFF: https://github.com/llvm/llvm-project/commit/45ab2b48bd55e50a86f6026ed31d2b60a118bdce.diff
LOG: [Clang] Improve the handling of large arrays evaluation.
This is a temporary fix (for clang 17) that caps the size of
any array we try to constant evaluate:
There are 2 limits:
* We cap to UINT_MAX the size of ant constant evaluated array,
because the constant evaluator does not support size_t.
* We cap to `-fconstexpr-steps` elements the size of each individual
array and dynamic array allocations.
This works out because the number of constexpr steps already limits
how many array elements can be initialized, which makes this new
limit conservatively generous.
This ensure that the compiler does not crash when attempting to
constant-fold valid programs.
If the limit is reached by a given array, constant evaluation will fail,
and the program will be ill-formed, until a bigger limit is given.
Or, constant folding will fail and the array will be evaluated at runtime.
Fixes #63562
Reviewed By: efriedma
Differential Revision: https://reviews.llvm.org/D155955
Added:
clang/test/SemaCXX/cxx2a-constexpr-dynalloc-limits.cpp
Modified:
clang/docs/ReleaseNotes.rst
clang/docs/UsersManual.rst
clang/include/clang/AST/Type.h
clang/include/clang/Basic/DiagnosticASTKinds.td
clang/lib/AST/ExprConstant.cpp
clang/lib/AST/Type.cpp
Removed:
################################################################################
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 98f50b428c2dc3..8ba3d22253a62b 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -112,6 +112,11 @@ Bug Fixes to Attribute Support
Bug Fixes to C++ Support
^^^^^^^^^^^^^^^^^^^^^^^^
+- Clang limits the size of arrays it will try to evaluate at compile time
+ to avoid memory exhaustion.
+ This limit can be modified by `-fconstexpr-steps`.
+ (`#63562 <https://github.com/llvm/llvm-project/issues/63562>`_)
+
Bug Fixes to AST Handling
^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 14af07d35f5e1d..a774fd91aa277f 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -3348,7 +3348,9 @@ Controlling implementation limits
.. option:: -fconstexpr-steps=N
Sets the limit for the number of full-expressions evaluated in a single
- constant expression evaluation. The default is 1048576.
+ constant expression evaluation. This also controls the maximum size
+ of array and dynamic array allocation that can be constant evaluated.
+ The default is 1048576.
.. option:: -ftemplate-depth=N
diff --git a/clang/include/clang/AST/Type.h b/clang/include/clang/AST/Type.h
index 8d20d088bb63c4..7526ec8906a32c 100644
--- a/clang/include/clang/AST/Type.h
+++ b/clang/include/clang/AST/Type.h
@@ -3144,6 +3144,8 @@ class ConstantArrayType final
QualType ElementType,
const llvm::APInt &NumElements);
+ unsigned getNumAddressingBits(const ASTContext &Context) const;
+
/// Determine the maximum number of active bits that an array's size
/// can require, which limits the maximum size of the array.
static unsigned getMaxSizeBits(const ASTContext &Context);
diff --git a/clang/include/clang/Basic/DiagnosticASTKinds.td b/clang/include/clang/Basic/DiagnosticASTKinds.td
index 566cdc3406058f..f73a289f80ca95 100644
--- a/clang/include/clang/Basic/DiagnosticASTKinds.td
+++ b/clang/include/clang/Basic/DiagnosticASTKinds.td
@@ -351,6 +351,9 @@ def note_constexpr_new_negative : Note<
"cannot allocate array; evaluated array bound %0 is negative">;
def note_constexpr_new_too_large : Note<
"cannot allocate array; evaluated array bound %0 is too large">;
+def note_constexpr_new_exceeds_limits : Note<
+ "cannot allocate array; evaluated array bound %0 exceeds the limit (%1); "
+ "use '-fconstexpr-steps' to increase this limit">;
def note_constexpr_new_too_small : Note<
"cannot allocate array; evaluated array bound %0 is too small to hold "
"%1 explicitly initialized elements">;
diff --git a/clang/lib/AST/ExprConstant.cpp b/clang/lib/AST/ExprConstant.cpp
index d9208b53e44994..c06ca3c405370e 100644
--- a/clang/lib/AST/ExprConstant.cpp
+++ b/clang/lib/AST/ExprConstant.cpp
@@ -1019,6 +1019,34 @@ namespace {
return false;
}
+ bool CheckArraySize(SourceLocation Loc, unsigned BitWidth,
+ uint64_t ElemCount, bool Diag) {
+ // FIXME: GH63562
+ // APValue stores array extents as unsigned,
+ // so anything that is greater that unsigned would overflow when
+ // constructing the array, we catch this here.
+ if (BitWidth > ConstantArrayType::getMaxSizeBits(Ctx) ||
+ ElemCount > uint64_t(std::numeric_limits<unsigned>::max())) {
+ if (Diag)
+ FFDiag(Loc, diag::note_constexpr_new_too_large) << ElemCount;
+ return false;
+ }
+
+ // FIXME: GH63562
+ // Arrays allocate an APValue per element.
+ // We use the number of constexpr steps as a proxy for the maximum size
+ // of arrays to avoid exhausting the system resources, as initialization
+ // of each element is likely to take some number of steps anyway.
+ uint64_t Limit = Ctx.getLangOpts().ConstexprStepLimit;
+ if (ElemCount > Limit) {
+ if (Diag)
+ FFDiag(Loc, diag::note_constexpr_new_exceeds_limits)
+ << ElemCount << Limit;
+ return false;
+ }
+ return true;
+ }
+
std::pair<CallStackFrame *, unsigned>
getCallFrameAndDepth(unsigned CallIndex) {
assert(CallIndex && "no call index in getCallFrameAndDepth");
@@ -3583,6 +3611,14 @@ static bool lifetimeStartedInEvaluation(EvalInfo &Info,
llvm_unreachable("unknown evaluating decl kind");
}
+static bool CheckArraySize(EvalInfo &Info, const ConstantArrayType *CAT,
+ SourceLocation CallLoc = {}) {
+ return Info.CheckArraySize(
+ CAT->getSizeExpr() ? CAT->getSizeExpr()->getBeginLoc() : CallLoc,
+ CAT->getNumAddressingBits(Info.Ctx), CAT->getSize().getZExtValue(),
+ /*Diag=*/true);
+}
+
namespace {
/// A handle to a complete object (an object that is not a subobject of
/// another object).
@@ -3757,6 +3793,9 @@ findSubobject(EvalInfo &Info, const Expr *E, const CompleteObject &Obj,
if (O->getArrayInitializedElts() > Index)
O = &O->getArrayInitializedElt(Index);
else if (!isRead(handler.AccessKind)) {
+ if (!CheckArraySize(Info, CAT, E->getExprLoc()))
+ return handler.failed();
+
expandArray(*O, Index);
O = &O->getArrayInitializedElt(Index);
} else
@@ -6491,6 +6530,9 @@ static bool HandleDestructionImpl(EvalInfo &Info, SourceLocation CallLoc,
uint64_t Size = CAT->getSize().getZExtValue();
QualType ElemT = CAT->getElementType();
+ if (!CheckArraySize(Info, CAT, CallLoc))
+ return false;
+
LValue ElemLV = This;
ElemLV.addArray(Info, &LocE, CAT);
if (!HandleLValueArrayAdjustment(Info, &LocE, ElemLV, ElemT, Size))
@@ -6681,7 +6723,7 @@ static bool HandleDestruction(EvalInfo &Info, SourceLocation Loc,
return HandleDestructionImpl(Info, Loc, LV, Value, T);
}
-/// Perform a call to 'perator new' or to `__builtin_operator_new'.
+/// Perform a call to 'operator new' or to `__builtin_operator_new'.
static bool HandleOperatorNewCall(EvalInfo &Info, const CallExpr *E,
LValue &Result) {
if (Info.checkingPotentialConstantExpression() ||
@@ -6727,13 +6769,12 @@ static bool HandleOperatorNewCall(EvalInfo &Info, const CallExpr *E,
return false;
}
- if (ByteSize.getActiveBits() > ConstantArrayType::getMaxSizeBits(Info.Ctx)) {
+ if (!Info.CheckArraySize(E->getBeginLoc(), ByteSize.getActiveBits(),
+ Size.getZExtValue(), /*Diag=*/!IsNothrow)) {
if (IsNothrow) {
Result.setNull(Info.Ctx, E->getType());
return true;
}
-
- Info.FFDiag(E, diag::note_constexpr_new_too_large) << APSInt(Size, true);
return false;
}
@@ -9617,14 +9658,12 @@ bool PointerExprEvaluator::VisitCXXNewExpr(const CXXNewExpr *E) {
// -- its value is such that the size of the allocated object would
// exceed the implementation-defined limit
- if (ConstantArrayType::getNumAddressingBits(Info.Ctx, AllocType,
- ArrayBound) >
- ConstantArrayType::getMaxSizeBits(Info.Ctx)) {
+ if (!Info.CheckArraySize(ArraySize.value()->getExprLoc(),
+ ConstantArrayType::getNumAddressingBits(
+ Info.Ctx, AllocType, ArrayBound),
+ ArrayBound.getZExtValue(), /*Diag=*/!IsNothrow)) {
if (IsNothrow)
return ZeroInitialization(E);
-
- Info.FFDiag(*ArraySize, diag::note_constexpr_new_too_large)
- << ArrayBound << (*ArraySize)->getSourceRange();
return false;
}
diff --git a/clang/lib/AST/Type.cpp b/clang/lib/AST/Type.cpp
index 99c859034423bb..cd1bcc4f17cdc6 100644
--- a/clang/lib/AST/Type.cpp
+++ b/clang/lib/AST/Type.cpp
@@ -175,6 +175,11 @@ unsigned ConstantArrayType::getNumAddressingBits(const ASTContext &Context,
return TotalSize.getActiveBits();
}
+unsigned
+ConstantArrayType::getNumAddressingBits(const ASTContext &Context) const {
+ return getNumAddressingBits(Context, getElementType(), getSize());
+}
+
unsigned ConstantArrayType::getMaxSizeBits(const ASTContext &Context) {
unsigned Bits = Context.getTypeSize(Context.getSizeType());
diff --git a/clang/test/SemaCXX/cxx2a-constexpr-dynalloc-limits.cpp b/clang/test/SemaCXX/cxx2a-constexpr-dynalloc-limits.cpp
new file mode 100644
index 00000000000000..0d26eca38ddcc5
--- /dev/null
+++ b/clang/test/SemaCXX/cxx2a-constexpr-dynalloc-limits.cpp
@@ -0,0 +1,95 @@
+// RUN: %clang_cc1 -std=c++20 -verify -fconstexpr-steps=1024 -Wvla %s
+
+namespace std {
+ using size_t = decltype(sizeof(0));
+}
+
+void *operator new(std::size_t, void *p) { return p; }
+
+namespace std {
+ template<typename T> struct allocator {
+ constexpr T *allocate(size_t N) {
+ return (T*)operator new(sizeof(T) * N); // #alloc
+ }
+ constexpr void deallocate(void *p) {
+ operator delete(p);
+ }
+ };
+ template<typename T, typename ...Args>
+ constexpr void construct_at(void *p, Args &&...args) { // #construct
+ new (p) T((Args&&)args...);
+ }
+}
+
+namespace GH63562 {
+
+template <typename T>
+struct S {
+ constexpr S(unsigned long long N)
+ : data(nullptr){
+ data = alloc.allocate(N); // #call
+ for(std::size_t i = 0; i < N; i ++)
+ std::construct_at<T>(data + i, i); // #construct_call
+ }
+ constexpr T operator[](std::size_t i) const {
+ return data[i];
+ }
+
+ constexpr ~S() {
+ alloc.deallocate(data);
+ }
+ std::allocator<T> alloc;
+ T* data;
+};
+
+constexpr std::size_t s = S<std::size_t>(1099511627777)[42]; // expected-error {{constexpr variable 's' must be initialized by a constant expression}} \
+ // expected-note@#call {{in call to 'this->alloc.allocate(1099511627777)'}} \
+ // expected-note@#alloc {{cannot allocate array; evaluated array bound 1099511627777 is too large}} \
+ // expected-note {{in call to 'S(1099511627777)'}}
+// Check that we do not try to fold very large arrays
+std::size_t s2 = S<std::size_t>(1099511627777)[42];
+std::size_t s3 = S<std::size_t>(~0ULL)[42];
+
+// We can allocate and initialize a small array
+constexpr std::size_t ssmall = S<std::size_t>(100)[42];
+
+// We can allocate this array but we hikt the number of steps
+constexpr std::size_t s4 = S<std::size_t>(1024)[42]; // expected-error {{constexpr variable 's4' must be initialized by a constant expression}} \
+ // expected-note@#construct {{constexpr evaluation hit maximum step limit; possible infinite loop?}} \
+ // expected-note@#construct_call {{in call}} \
+ // expected-note {{in call}}
+
+
+
+constexpr std::size_t s5 = S<std::size_t>(1025)[42]; // expected-error{{constexpr variable 's5' must be initialized by a constant expression}} \
+ // expected-note@#alloc {{cannot allocate array; evaluated array bound 1025 exceeds the limit (1024); use '-fconstexpr-steps' to increase this limit}} \
+ // expected-note@#call {{in call to 'this->alloc.allocate(1025)'}} \
+ // expected-note {{in call}}
+
+
+// Check we do not perform constant initialization in the presence
+// of very large arrays (this used to crash)
+
+template <auto N>
+constexpr int stack_array() {
+ [[maybe_unused]] char BIG[N] = {1}; // expected-note 3{{cannot allocate array; evaluated array bound 1025 exceeds the limit (1024); use '-fconstexpr-steps' to increase this limit}}
+ return BIG[N-1];
+}
+
+int a = stack_array<~0U>();
+int c = stack_array<1024>();
+int d = stack_array<1025>();
+constexpr int e = stack_array<1024>();
+constexpr int f = stack_array<1025>(); // expected-error {{constexpr variable 'f' must be initialized by a constant expression}} \
+ // expected-note {{in call}}
+void ohno() {
+ int bar[stack_array<1024>()];
+ int foo[stack_array<1025>()]; // expected-warning {{variable length arrays are a C99 feature}} \
+ // expected-note {{in call to 'stack_array()'}}
+
+ constexpr int foo[stack_array<1025>()]; // expected-warning {{variable length arrays are a C99 feature}} \
+ // expected-error {{constexpr variable cannot have non-literal type 'const int[stack_array<1025>()]'}} \
+ // expected-note {{in call to 'stack_array()'}}
+}
+
+}
More information about the cfe-commits
mailing list