<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 29, 2016, at 11:22 AM, Mehdi Amini via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><br class="Apple-interchange-newline">On Nov 29, 2016, at 9:14 AM, Reid Kleckner via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">I think that all makes sense. You're just adding the missing non-ODR conterpart of 'external' linkage. I could imagine having "external / external_odr" linkage for example.<div class=""><br class=""></div><div class="">That said, do you think we should take the opportunity to split out a bit for interposability so that we can kill off the *_odr linkage variants? Today's non-ODR weak functions would look more like this:</div><div class=""> define weak interposable void @foo() { ret void }</div><div class=""><br class=""></div><div class="">We could probably preserve bitcode compatibility by continuing to use the old combined linkage encoding. This is useful because we want the old weak to decode as weak+interposable and the old weak_odr to decode as weak.</div><div class=""><div class=""><br class=""></div><div class="">Some more prior discussion:<span class="Apple-converted-space"> </span><a href="https://reviews.llvm.org/D19995#423481" class="">https://reviews.llvm.org/D19995#423481</a></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">See also the open PR: <a href="https://llvm.org/bugs/show_bug.cgi?id=23501" class="">https://llvm.org/bugs/show_bug.cgi?id=23501</a></div></div></div></blockquote><div><br class=""></div><div>And another prior discussion and patch by Rafael (CC): <a href="https://reviews.llvm.org/D20217" class="">https://reviews.llvm.org/D20217</a></div><div><br class=""></div><div>— </div><div>Mehdi</div><div><br class=""></div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class=""><br class=""></div>— </div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Mehdi</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, Nov 29, 2016 at 8:01 AM, Hal Finkel via llvm-dev<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div class=""><div class="" style="font-family: arial, helvetica, sans-serif; font-size: 10pt;">Hi everyone,<br class=""><br class="">Clang/LLVM's support for ELF interposition is in a confusing state, and I propose making a few (hopefully simple) adjustments in order to bring our model into a self-consistent state.<br class=""><br class="">The problem: On ELF systems, global symbols can be interposed. This means, for example, that calls to global functions in some (shared) library defined in that same library might end up being redirected to an implementation in some other library (or in the main executable). The most common reason for this is the use of LD_PRELOAD, but there are plenty of other ways to trigger interposition as well. As a result, it is technically inconsistent to inline any global function or do inter-procedural analysis on them because the implementation might be replaced by code with completely different behavior at runtme (or link time). Clang has never supported this (i.e. we do treat these functions as being eligible for inlining and perform IPA on them). GCC, on the other hand, has traditionally respected the possibility of ELF interposition and refrained from doing these things (at least when compiling with -fPIC).<br class=""><br class="">I believe that Clang/LLVM's current behavior is the most-useful behavior and we should keep the current behavior (at least as a default). I do understand, however, that there are valid use cases for ELF interposition and places where we should allow it (e.g. when compiling certain system libraries). GCC recently added a flag -fsemantic-interposition/-fno-<wbr class="">semantic-interposition, where using -fno-semantic-interposition provides Clang/LLVM's behavior of assuming that ELF interposition will not be used.<br class=""><br class="">It has been suggested that, to be self consistent, LLVM should emit global symbols with protected ELF visibility in cases where we've assumed that ELF interposition won't happen. ELF protected visibility does seem to have exactly that meaning: A protected global symbol is externally visible but cannot be interposed. Unfortunately, as I understand it, on some major platforms (e.g. x86), protected-visibility symbols have a major flaw: Non-uniqueness of function pointers (i.e. the function pointer obtained to a function outside of the defining library might be different from the pointer obtained within the defining library). As a result, making this change might be practically prohibited (even if it makes sense in theory).<br class=""><br class="">Proposal:<br class=""><br class=""> 1. Add a new linkage type, interposible, which is like external except that isInterposableLinkage will return true (thus preventing inlining, IPA, etc.). This is similar to weak linkage, in a sense, except that such symbols are never discarded and are not marked as weak for linking, etc.<br class=""><br class=""> 2. Add -fsemantic-interposition/-fno-<wbr class="">semantic-interposition to Clang. Default to -fno-semantic-interposition, but when -fsemantic-interposition is used, use interposible linkage for all functions where external linkage might otherwise have been used.<br class=""><br class="">Thoughts?<br class=""><br class="">Some useful links:<br class=""><a href="http://hubicka.blogspot.com/2015/04/GCC5-IPA-LTO-news.html" target="_blank" class="">http://hubicka.blogspot.com/<wbr class="">2015/04/GCC5-IPA-LTO-news.html</a><span class="Apple-converted-space"> </span>(the section on the -fno-semantic-interposition flag)<br class=""><a href="https://gcc.gnu.org/ml/gcc-patches/2014-05/msg01671.html" target="_blank" class="">https://gcc.gnu.org/ml/gcc-<wbr class="">patches/2014-05/msg01671.html</a><br class=""><br class="">On some issues with ELF protected-visibility symbols:<br class=""><a href="http://www.macieira.org/blog/2012/01/sorry-state-of-dynamic-libraries-on-linux/" target="_blank" class="">http://www.macieira.org/blog/<wbr class="">2012/01/sorry-state-of-<wbr class="">dynamic-libraries-on-linux/</a><br class=""><a href="http://www.airs.com/blog/archives/307" target="_blank" class="">http://www.airs.com/blog/<wbr class="">archives/307</a><br class=""><a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19520" target="_blank" class="">https://gcc.gnu.org/bugzilla/<wbr class="">show_bug.cgi?id=19520</a><br class=""><br class="">Thanks again,<br class="">Hal<br class=""><br class="">P.S. For some previous discussion on this, see below...<br class=""><br class=""><hr id="m_-5203822644475916653zwchr" class=""><blockquote class="" style="border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-size: 12pt;"><b class="">From:<span class="Apple-converted-space"> </span></b>"Hal Finkel via llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>><br class=""><b class="">To:<span class="Apple-converted-space"> </span></b>"James Y Knight" <<a href="mailto:jyknight@google.com" target="_blank" class="">jyknight@google.com</a>><br class=""><b class="">Cc:<span class="Apple-converted-space"> </span></b>"llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>><br class=""><b class="">Sent:<span class="Apple-converted-space"> </span></b>Monday, February 29, 2016 9:50:15 AM<br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>Re: [llvm-dev] Possible soundness issue with available_externally (split from "RFC: Add guard intrinsics")<br class=""><br class=""><br class=""><div class="" style="font-family: arial, helvetica, sans-serif; font-size: 10pt;"><hr id="m_-5203822644475916653zwchr" class=""><blockquote class="" style="border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-size: 12pt;"><b class="">From:<span class="Apple-converted-space"> </span></b>"James Y Knight" <<a href="mailto:jyknight@google.com" target="_blank" class="">jyknight@google.com</a>><br class=""><b class="">To:<span class="Apple-converted-space"> </span></b>"Hal Finkel" <<a href="mailto:hfinkel@anl.gov" target="_blank" class="">hfinkel@anl.gov</a>><br class=""><b class="">Cc:<span class="Apple-converted-space"> </span></b>"Sanjoy Das" <<a href="mailto:sanjoy@playingwithpointers.com" target="_blank" class="">sanjoy@playingwithpointers.<wbr class="">com</a>>, "llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>><br class=""><b class="">Sent:<span class="Apple-converted-space"> </span></b>Monday, February 29, 2016 9:31:24 AM<br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>Re: [llvm-dev] Possible soundness issue with available_externally (split from "RFC: Add guard intrinsics")<br class=""><br class=""><div dir="ltr" class=""><p dir="ltr" class="">On Feb 26, 2016 8:50 PM, "Hal Finkel" <<a href="mailto:hfinkel@anl.gov" target="_blank" class="">hfinkel@anl.gov</a>> wrote:<br class=""></p><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class="" style="font-family: arial, helvetica, sans-serif; font-size: 10pt;"><blockquote class="" style="border-left-width: 2px; border-left-style: solid; border-left-color: rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-size: 12pt;"><b class="">From:<span class="Apple-converted-space"> </span></b>"James Y Knight via llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>><br class=""><b class="">To:<span class="Apple-converted-space"> </span></b>"Sanjoy Das" <<a href="mailto:sanjoy@playingwithpointers.com" target="_blank" class="">sanjoy@playingwithpointers.<wbr class="">com</a>><br class=""><b class="">Cc:<span class="Apple-converted-space"> </span></b>"llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>><br class=""><b class="">Sent:<span class="Apple-converted-space"> </span></b>Thursday, February 25, 2016 1:41:43 PM<br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>Re: [llvm-dev] Possible soundness issue with available_externally (split from "RFC: Add guard intrinsics")<br class=""><br class=""><div dir="ltr" class="">While we're talking about this, I'd just mention again that the same issue arises for *normal* functions too, when linked into a shared library:<div class=""> int foo() { return 1; }<br class=""></div><div class=""> int bar() { return foo(); }<br class=""></div><div class=""><br class=""></div><div class="">Now, compare:</div><div class=""> clang -fPIC -O1 -S -o - test.c<br class=""></div><div class=""> gcc -fPIC -O1 -S -o - test.c<br class=""></div><div class=""><br class=""></div><div class="">GCC will refuse to inline foo into bar, or use any information about foo in compiling bar, because foo is exported in the dynamic symbol table, and thus replaceable via symbol interposition.</div><div class=""><br class=""></div><div class="">Clang assumes that you won't do that, or that you don't care what happens if you do. It will happily inline. And, in absense of inlining (e.g. if foo is too long to inline), clang will deduce function attributes about foo and rely on those in bar -- despite that the call goes through the PLT and could in fact be an entirely different unrelated implementation (or, for that matter, a differently-optimized version of the same implementation).</div><div class=""><br class=""></div><div class="">Is that *really* okay?</div><div class=""><br class=""></div></div></blockquote>I'm comfortable with saying that symbol interposition falls outside of the model we have for the targeted system (at least by default), and thus, this is okay. We also don't model the possibility of someone hex-editing the binary ;)</div></div></blockquote><div class=""><br class=""></div><div class=""><p dir="ltr" class="">I'm not really okay with it; the current behavior feels unprincipled.</p><p dir="ltr" class="">We have a visibility attribute which can be used to control this: On ELF systems, "default" visibililty allows interposition (unlike on Darwin) -- that is, it explicitly ALLOWS for replacing the symbol's definition. The policy of "You can't replace the definition of the symbol, but it is globally visible" is exactly what the "protected" visibility mode is for.<br class=""></p><p dir="ltr" class="">If we want to say that you can't interpose by default on ELF targets, that would be a choice. Then, we should make the default symbol visibility "protected" instead of "default". But, continuing to generate calls through the PLT -- which is only needed because the symbols might be replaced -- while simultaneously making optimizations that are broken if they actually ARE replaced, seems kinda bogus.</p></div></div></div></blockquote>This makes sense, and I think you understand my concern here: Most programmers don't understand these issues, nor do they ever expect to use dynamic interposition. They do expect, however, that the compiler has good IPA and will use the information it is provided effectively. I'd be happy to make the default visibility protected, allowing us to continue optimizing well, and provide a principled behavior otherwise. Given, as you point out, this is the default on Darwin, is there experience from Darwin porting, or any other factors, that would indicate this would be a hardship?<br class=""><br class="">Thanks again,<br class="">Hal<span class="HOEnZb"><font color="#888888" class=""><br class=""><br class="">--<span class="Apple-converted-space"> </span><br class=""><div class=""><span class=""></span>Hal Finkel<br class="">Assistant Computational Scientist<br class="">Leadership Computing Facility<br class="">Argonne National Laboratory<span class=""></span><br class=""></div></font></span></div><span class="HOEnZb"><font color="#888888" class=""><br class="">______________________________<wbr class="">_________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank" class="">http://lists.llvm.org/cgi-bin/<wbr class="">mailman/listinfo/llvm-dev</a><br class=""></font></span></blockquote><span class="HOEnZb"><font color="#888888" class=""><br class=""><br class=""><br class="">--<span class="Apple-converted-space"> </span><br class=""><div class=""><span name="x" class=""></span>Hal Finkel<br class="">Lead, Compiler Technology and Programming Languages<br class="">Leadership Computing Facility<br class="">Argonne National Laboratory<span name="x" class=""></span><br class=""></div></font></span></div></div><br class="">______________________________<wbr class="">_________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/<wbr class="">mailman/listinfo/llvm-dev</a><br class=""><br class=""></blockquote></div><br class=""></div>_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class=""></div></blockquote></div><br class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">LLVM Developers mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">llvm-dev@lists.llvm.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></body></html>