Anonymous records with typedef names for linkage

[John McCall]

Our current approach to computing linkage is pretty badly broken in the presence of anonymous types that gain linkage through the timely use of a typedef, e.g.:
  typedef struct {
    int foo() { … }
  } A;

For the entirety of the definition of this struct, we will consider it to not be externally visible because it’s anonymous; but as soon as we process the declarator, it gains external linkage.  This is problematic.  r200380 changes two pieces of code that only really apply to file-scope declarations to actually filter for that before querying linkage, and that’s a win on multiple axes, but it’s not a workable approach to avoiding all such crashes because of the ‘this’ problem.

The ‘this’ problem is quite simple.  For the most part, within the definition of an anonymous struct (including an anonymous-so-far struct like this one), we are protected from the struct type escaping into truly arbitrary parts of the compiler by the fact that you can’t actually refer to it directly.  This almost makes it possible to just carefully control linkage queries during the definition of a struct, like r200380 does, and thereby avoid any semantic troubles.  But it’s not actually possible, because you can define a non-static member function in such a type, and such a function can name its containing type by using ‘this’: in C++11, you can just use decltype(*this), and in C++03, you can use template argument deduction to bind a template argument to the type (*).  Either feature allows the type to escape into the broader environment, and it becomes impossible for us to code around the problem.

(*) This is, of course, ill-formed prior to C++11 because the type doesn’t have external linkage, except that it isn’t ill-formed if the anonymous struct acquires a typedef name for linkage purposes.

As far as I can tell, this is not technically a language problem, because there is nothing in the language which requires us to compute linkage at a specific point during the processing of a file.  For example, C++03 requires us to check whether a template argument has external linkage, but there’s nothing forcing us to check that while processing the template argument; we could delay the check indefinitely, even to the end of the translation unit, as long as we eventually diagnose it.  But I’m reluctant to actually propose that we pursue an approach that delays all linkage computations that might involve a still-being-defined anonymous declaration until that declaration is complete; that seems like a very brittle solution.  And yet I don’t have a better proposal that’s totally correct.

Unfortunately, it’s not possible to prove one way or the other whether a given anonymous declaration will later be given a typedef name for the purposes of linkage.  While nobody actually codes this way, the ‘typedef’ keyword is allowed to appear after the class-specifier, even if the class-specifier contains a definition; and for the inverse, a ‘typedef’ declaration does not create a typedef name for linkage purposes if none of the declarators is just a simple name (e.g. if it’s "typedef struct { … } *x;”).

I think the simplest solution for now is to just check for cached linkage before setting a typedef name for linkage purposes and then diagnosing it as an unsupported feature.

John.