[PATCH] D66361: Improve behavior in the case of stack exhaustion.

[Richard Smith - zygoloid via Phabricator via cfe-commits]

rsmith created this revision.
rsmith added reviewers: rnk, aaron.ballman.
Herald added subscribers: cfe-commits, mgorny.
Herald added a project: clang.

Clang performs various recursive operations (such as template instantiation),
and may use non-trivial amounts of stack space in each recursive step (for
instance, due to recursive AST walks). While we try to keep the stack space
used by such steps to a minimum and we have explicit limits on the number of
such steps we perform, it's impractical to guarantee that we won't blow out the
stack on deeply recursive template instantiations on complex ASTs, even with
only a moderately high instantiation depth limit.

The user experience in these cases is generally terrible: we crash with
no hint of what went wrong. Under this patch, we attempt to do better:

- Detect when the stack is nearly exhausted, and produce a warning with a nice template instantiation backtrace, telling the user that we might run slowly or crash.
- For cases where we're forced to trigger recursive template instantiation in arbitrarily-deeply-nested contexts, check whether we're nearly out of stack space and allocate a new stack (by spawning a new thread) after producing the warning.


Repository:
  rC Clang

https://reviews.llvm.org/D66361

Files:
  include/clang/Basic/DiagnosticSemaKinds.td
  include/clang/Basic/Stack.h
  include/clang/Sema/Sema.h
  lib/Basic/CMakeLists.txt
  lib/Basic/Stack.cpp
  lib/Frontend/CompilerInstance.cpp
  lib/Sema/Sema.cpp
  lib/Sema/SemaExpr.cpp
  lib/Sema/SemaInit.cpp
  lib/Sema/SemaLookup.cpp
  lib/Sema/SemaTemplate.cpp
  lib/Sema/SemaTemplateDeduction.cpp
  lib/Sema/SemaTemplateInstantiate.cpp
  lib/Sema/SemaTemplateInstantiateDecl.cpp
  lib/Sema/SemaType.cpp
  test/SemaTemplate/stack-exhaustion.cpp
  tools/driver/driver.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D66361.215657.patch
Type: text/x-patch
Size: 30470 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20190816/91f22675/attachment-0001.bin>