[cfe-dev] Clang crash on infinite template instantiation

[Pan, Wei]

Hi Richard,

Disabling BuildRecoveryCallExpr seems correct to solving this problem. Do you think this patch is correct?

Thanks!

Wei

From: metafoo at gmail.com [mailto:metafoo at gmail.com] On Behalf Of Richard Smith
Sent: Thursday, September 06, 2012 6:26 PM
To: Pan, Wei
Cc: cfe-dev at cs.uiuc.edu
Subject: Re: [cfe-dev] Clang crash on infinite template instantiation

On Thu, Sep 6, 2012 at 1:50 PM, Pan, Wei <wei.pan at intel.com<mailto:wei.pan at intel.com>> wrote:
Hello Clang Dev,

Recently I looked into the clang bug 12053,

template <typename T> auto foo(T t) -> decltype(foo(t)) {}

http://llvm.org/bugs/show_bug.cgi?id=12053

which crashes clang (trunk)  (and gcc 4.6 too).

As far as I know,  Clang does realize that there is no candidate
available while resolving "decltype(foo(t))", however BuildRecoveryCallExpr
will find  template foo (in DiagnoseTwoPhaseLookup)
and try to instantiate it again. This leads to the crash.

I am wondering if the following is the right way to fix this. The basic idea is:

Before starting instantiating a function template, we check if the same function
template instantiation *with the same template arguments* is already in-progress.
If yes, then clang is not making any progress and should lead an infinite loop.
We treat it as an SFINAE.

The attached patch will fix the clang crashing on the above test and other similar tests like

template <typename T> auto foo(T t) -> decltype(bar(t)) {}
template <typename T> auto bar(T t) -> decltype(foo(t)) {}
int x = foo(0);

This is not a final patch since this change will affect two tests (only these two)
instantiation-depth-subst.cpp and instantiation-depth-subst-2.cpp.
Any thoughts?

I generally like the idea of checking for cyclic function template instantiations, but I'm hesitant about this approach -- scanning through the instantiation stack each time introduces overhead which grows quadratically in the instantiation depth, which could be too slow for deep instantiation stacks.

Also, this approach doesn't solve the entire problem. For instance, in this case, there is no cycle:

  template<typename T> auto foo(T t) -> decltype(foo(&t)) {}

Perhaps we can solve this more directly, by just disabling BuildRecoveryCallExpr when we're already in the middle of recovery.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20120907/596857ec/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a.patch
Type: application/octet-stream
Size: 4438 bytes
Desc: a.patch
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20120907/596857ec/attachment.obj>