<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Jul 11, 2018 at 5:29 PM, Louis Dionne <span dir="ltr"><<a href="mailto:ldionne@apple.com" target="_blank">ldionne@apple.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space"><div>- llvm-dev (sorry Chandler, I’m not accustomed to which topics should be discussed on which lists yet)</div><br><div><div><div class="h5"><blockquote type="cite"><div>On Jul 11, 2018, at 06:37, Chandler Carruth <<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>> wrote:</div><br class="m_-4682466315935232800Apple-interchange-newline"><div><div dir="ltr">This is probably much more of a question for the Clang list....<br><br><div class="gmail_quote"><div dir="ltr">On Tue, Jul 10, 2018 at 6:21 PM Hubert Tong via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Jul 10, 2018 at 7:20 PM, Louis Dionne via llvm-dev <span dir="ltr"><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
While investigating the situation of visibility annotations and linkage in libc++ with the goal of removing uses of `__always_inline__`, Eric Fiselier and I stumbled upon the attached test case, which I don't think Clang compiles properly. Here's the gist of the test case, reduced to the important parts (see the attachment if you want to repro):<br>
<br>
// RUN: %cxx -shared -o %T/libtest.so %flags %compile_flags -fPIC %s<br>
// RUN: %cxx -c -o %T/main.o %flags %compile_flags %s -O2 -DBUILDING_MAIN<br>
// RUN: %cxx -o %T/test.exe -L%T/ -Wl,-rpath,%T/ -ltest %T/main.o<br>
// RUN: %T/test.exe<br>
<br>
template <class T><br>
struct Foo {<br>
inline __attribute__((visibility("<wbr>hidden")))<br>
int __init(int x) { /* LOTS OF CODE */ }<br>
<br>
inline __attribute__((visibility("<wbr>default")))<br>
int bar(int x) { return __init(x); }<br>
};<br>
<br>
extern template struct Foo<int>;<br>
<br>
#ifdef BUILDING_MAIN<br>
int main() {<br>
Foo<int> f;<br>
f.bar(101);<br>
}<br>
#else<br>
template struct Foo<int>;<br>
#endif<br>
<br>
When running the attached file in `lit`, we get:<br>
<br>
Undefined symbols for architecture x86_64:<br>
"Foo<int>::__init(int)", referenced from:<br>
_main in main.o<br>
ld: symbol(s) not found for architecture x86_64<br>
<br>
The idea here is that `__init` is a pretty big function, and we're promising that an external definition of it is available through the use of the extern template declaration. With the appropriate optimization level (O2 and above), LLVM decides not to include the definition of `__init` in the executable and to use the one available externally. Unfortunately, `__init` has hidden visibility, and so the definition in the .so is not visible to the executable, and the link fails.<br>
<br>
Where I think Clang/LLVM goes wrong is when it decides to remove the instantiation in the executable in favor of the one that is hypothetically provided externally. Indeed, the Standard says in [temp.explicit]/12 (<a href="http://eel.is/c++draft/temp.explicit#12" rel="noreferrer" target="_blank">http://eel.is/c++draft/temp.<wbr>explicit#12</a>):<br>
<br>
Except for inline functions and variables, declarations with types deduced from their initializer or return value ([dcl.spec.auto]), const variables of literal types, variables of reference types, and class template specializations, explicit instantiation declarations have the effect of suppressing the implicit instantiation of the definition of the entity to which they refer. [ Note: The intent is that an inline function that is the subject of an explicit instantiation declaration will still be implicitly instantiated when odr-used so that the body can be considered for inlining, but that no out-of-line copy of the inline function would be generated in the translation unit. — end note ]<br>
<br>
Only reading the normative wording, it seems like LLVM should leave the instantiation there because it can't actually assume that there will be a definition provided elsewhere (yes, despite the extern template declaration, because the function is inline). Then, the non-normative note seems to be approving of what LLVM is doing, but I'm wondering whether that's really the intended behavior.<br>
<br>
Questions:<br>
1. Is what LLVM's doing there legal?<br></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>The Standard does not say anything about the instantiation producing a definition associated with object file that results from translating the current translation unit. The program is only valid if there is indeed an explicit instantiation provided elsewhere. That said, the as-if rule that allows the suppression of the definition assumes that a provided explicit instantiation can be linked against. It is up the the designers of the extension (the visibility attribute) to say whether or not the rule with templates having hidden visibility is that the explicit instantiation needs to be provided in the same “module".<br></div></div></div></div></blockquote></div></div></div></blockquote><div><br></div></div></div><div>Thanks Hubert. That makes some amount of sense. So in that case, it would either be</div><div>1. A bug that Clang is not emitting `__init` in the object file since it has hidden visibility, and it can’t tell whether we’re going to try to link dynamically or statically with the other module that provides it.</div><div>2. The intended design that visibility attributes interact very poorly with extern template declarations. This would be sad, since libc++ is a pretty big user of both, and it really isn’t working that well for us.</div></div></div></blockquote><div>Let's map visibility attributes as affecting what constitutes the sets of eligible uses and definitions where the ODR and the similar rules around templates apply. Which is to say, it partitions your program.<br></div><div>For example, it is okay to have multiple definitions of non-inline void ::foo(); having external linkage and hidden or internal visibility as long as each definition was in a separate module.<br></div><div>Similarly, it is not okay have the only definition of void ::bar() be hidden or internal in one module when uses occur in another.<br></div><div>This model leads to the current result that is observed with extern template declarations.<br><br></div><div>Technically, a possible solution is to provide explicit instantiation definitions with inline visibility in an object file to be included on the link step.<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space"><div><div><br></div><div>In both cases, it’s not a conformance bug with the Standard, as Hubert points out. It would be lovely if someone more knowledgeable about the visibility attributes could shed some light on whether that’s intended and we need to invent something new, or whether it is correct to consider it a bug and we should fix it (somehow).</div><div><br></div><div>In the meantime, this is blocking our ability to use neither `__attribute__((Internal_<wbr>linkage))` nor `__attribute__((__always_<wbr>inline__))` on any declaration that may have an extern template declaration (which is basically all declarations, because users can write extern template declarations for std:: components in their own code).</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Louis</div></font></span><span class=""><br><blockquote type="cite"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
2. If it is legal, then libc++ needs a way to express that a function should either be inlined in the caller, or emitted in the TU and de-duplicated later (I think that’s linkonce_odr), despite there being an extern template declaration promising a definition elsewhere. I think the current situation is that the function gets available_externally linkage instead. Is there a way to express this at the C++ source code level?<br>
<br>
Thank you,<br>
Louis Dionne<br>
<br>
<br>______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
<br></blockquote></div></div></div>
______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
</blockquote></div></div>
</blockquote></span></div><br></div></blockquote></div><br></div></div>