[LLVMdev] -fvisibility=hidden, and typeinfo, and type-erasure

[Akim Demaille]

[Was initially posted on cfe-users, sorry.]

Hi,

I'm sorry my message is quite long, the TL;DR version is "g++ and clang++ seem to have different opinions on how RTTI, templates, and ELF visibility should interact".



I can't tell whether this is a bug or not: I have found no relevant documentation that could help me decide whether this behavior is meant, or not.  All I can say is that the current behavior is not the one I would expect, but maybe you guys have a different opinion, which I'd be happy to hear about.  To my eyes it looks like a violation of the One Definition Rule, but since ELF visibility issues are not covered by the standard, this is wishful thinking :)

I'm reproducing here basically what I had written there:

    http://stackoverflow.com/questions/19496643/

-------------------------------------------------

This is a scaled down version of a problem I am facing with clang++ on Mac OS X.

The failure
===========

I have this big piece of software in C++ with a large set of symbols in the object files, so I'm using `-fvisibility=hidden` to keep my symbol tables small.  It is well known that in such a case one must pay extra attention to the vtables, and I suppose I face this problem.  I don't know however, how to address it elegantly in a way that pleases both gcc and clang.

Consider a `base` class which features a down-casting operator, `as`, and a `derived` class template, that contains some payload.  The pair `base`/`derived<T>` is used to implement type-erasure:

   // foo.hh

   #define API __attribute__((visibility("default")))

   struct API base
   {
     virtual ~base() {}

     template <typename T>
     const T& as() const
     {
       return dynamic_cast<const T&>(*this);
     }
   };

   template <typename T>
   struct API derived: base
   {};

   struct payload {}; // *not* flagged as "default visibility".

   API void bar(const base& b);
   API void baz(const base& b);


Then I have two different compilation units that provide a similar service, which I can approximate as twice the same feature: down-casting from `base` to `derive<payload>`:

   // bar.cc
   #include "foo.hh"
   void bar(const base& b)
   {
     b.as<derived<payload>>();
   }

and

   // baz.cc
   #include "foo.hh"
   void baz(const base& b)
   {
     b.as<derived<payload>>();
   }

>From these two files, I build a dylib.  Here is the `main` function, calling these functions from the dylib:

   // main.cc
   #include <stdexcept>
   #include <iostream>
   #include "foo.hh"

   int main()
   try
     {
       derived<payload> d;
       bar(d);
       baz(d);
     }
   catch (std::exception& e)
     {
       std::cerr << e.what() << std::endl;
     }

Finally, a Makefile to compile and link everybody.  Nothing special here, except, of course, `-fvisibility=hidden`.

   CXX = clang++
   CXXFLAGS = -std=c++11 -fvisibility=hidden

   all: main

   main: main.o bar.dylib baz.dylib
   	$(CXX) -o $@ $^

   %.dylib: %.cc foo.hh
   	$(CXX) $(CXXFLAGS) -shared -o $@ $<

   %.o: %.cc foo.hh
   	$(CXX) $(CXXFLAGS) -c -o $@ $<

   clean:
   	rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib

The run succeeds with gcc (4.8 and 4.9) on OS X:

   $ make clean && make CXX=g++-mp-4.8 && ./main 
   rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -c main.cc -o main.o
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
   g++-mp-4.8 -o main main.o bar.dylib baz.dylib

However with clang (3.4 and 3.5), this fails (the typeids have different addresses):

   $ make clean && make CXX=clang++-mp-3.4 && ./main
   rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -c main.cc -o main.o
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
   clang++-mp-3.4 -o main main.o bar.dylib baz.dylib
   std::bad_cast

However it works if I tag my payload with a public visibility:

   struct API payload {};

but I do not want to expose the payload type.  So my questions are:

1. why are GCC and Clang different here?
2. is it _really_ working with GCC, or I was just "lucky" in my use of undefined behavior?
3. do I have a means to avoid making `payload` go public with Clang++?

Thanks in advance.

Type equality of visible class templates with invisible type parameters (EDIT)
==============================================================================

I have now a better understanding of what is happening.  It is appears that both GCC _and_ clang require both the class template and its parameter to be visible (in the ELF sense) to build a unique type.  If you change the `bar.cc` and `baz.cc` functions as follows:

   // bar.cc
   #include "foo.hh"
   void bar(const base& b)
   {
     std::cerr
       << "bar value: " << &typeid(b) << std::endl
       << "bar type:  " << &typeid(derived<payload>) << std::endl
       << "bar equal: " << (typeid(b) == typeid(derived<payload>)) << std::endl;
     b.as<derived<payload>>();
   }

and *if* you make `payload` visible too:

   struct API payload {};

then both GCC and Clang succeed (same typeid address):

   $ make clean && make CXX=g++-mp-4.8
   rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -c -o main.o main.cc
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
   ./g++-mp-4.8 -o main main.o bar.dylib baz.dylib
   $ ./main
   bar value: 0x106785140
   bar type:  0x106785140
   bar equal: 1
   baz value: 0x106785140
   baz type:  0x106785140
   baz equal: 1

   $ make clean && make CXX=clang++-mp-3.4
   rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -c -o main.o main.cc
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
   clang++-mp-3.4 -o main main.o bar.dylib baz.dylib
   $ ./main
   bar value: 0x10a6d5110
   bar type:  0x10a6d5110
   bar equal: 1
   baz value: 0x10a6d5110
   baz type:  0x10a6d5110
   baz equal: 1

Type equality is easy to check, there is actually a single instantiation of the type, as witnessed by its unique address.

However, if you remove the visible attribute from `payload`:

   struct payload {};

then you get with GCC:

   $ make clean && make CXX=g++-mp-4.8
   rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -c -o main.o main.cc
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
   g++-mp-4.8 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
   g++-mp-4.8 -o main main.o bar.dylib baz.dylib
   $ ./main
   bar value: 0x10faea120
   bar type:  0x10faf1090
   bar equal: 1
   baz value: 0x10faea120
   baz type:  0x10fafb090
   baz equal: 1

Now there are several instantiation of the type `derived<payload>` (as witnessed by the three different addresses), but GCC sees these types are equal, and (of course) the two `dynamic_cast` pass.

In the case of clang, it's different:

   $ make clean && make CXX=clang++-mp-3.4
   rm -f main main.o bar.o baz.o bar.dylib baz.dylib libba.dylib
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -c -o main.o main.cc
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o bar.dylib bar.cc
   clang++-mp-3.4 -std=c++11 -fvisibility=hidden -shared -o baz.dylib baz.cc
   .clang++-mp-3.4 -o main main.o bar.dylib baz.dylib
   $ ./main
   bar value: 0x1012ae0f0
   bar type:  0x1012b3090
   bar equal: 0
   std::bad_cast

There are also three instantiations of the type (removing the failing `dynamic_cast` does show that there are three), but this time, they are not equal, and the `dynamic_cast` (of course) fails.

Now the question turns into:
1. is this difference between both compilers wanted by their authors
2. if not, what is "expected" behavior between both

I prefer GCC's semantics, as it allows to really implement type-erasure without any need to expose publicly the wrapped types.