<div dir="ltr">Thanks Peter, your patch does fix the reproducer. I filed <a href="https://bugs.llvm.org/show_bug.cgi?id=37422">https://bugs.llvm.org/show_bug.cgi?id=37422</a> to track this bug. I have no clue on how to resolve the tests - whether further cleanup is required in the code or in the tests. But if Teresa or you cannot get to it, I can, with some help, take a crack at fixing the tests.</div><br><div class="gmail_quote"><div dir="ltr">On Wed, May 9, 2018 at 11:26 AM Peter Collingbourne <<a href="mailto:peter@pcc.me.uk">peter@pcc.me.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The problem is that ThinLTO is not dropping the non-prevailing definitions, and they end up being emitted into the object file for b.o.<div><br><div><div>$ ../ra/bin/llvm-dis -o - b.o0.0.preopt.bc | grep __llvm_prof</div><div>$__llvm_profile_raw_version = comdat any</div><div>$__llvm_profile_filename = comdat any</div><div>@__llvm_profile_raw_version = constant i64 72057594037927940, comdat</div><div>@__llvm_profile_filename = constant [19 x i8] c"default_%m.profraw\00", comdat</div></div><div><br></div><div>lld ignores comdats in LTO object files because it expects all comdats to have already been resolved. So we see this error. </div><div><br></div><div>We are supposed to drop non-prevailing symbols here, but for some reason we do it only for weak symbols. That's not correct in ELF where comdat symbols can be both strong and non-prevailing.</div><div><a href="http://llvm-cs.pcc.me.uk/lib/LTO/LTO.cpp#290" target="_blank">http://llvm-cs.pcc.me.uk/lib/LTO/LTO.cpp#290</a><br></div><div><br></div><div>This patch fixes the reproducer but it leads to other test failures that would need to be looked at.</div></div><div><div><br></div><div>diff --git a/llvm/lib/LTO/LTO.cpp b/llvm/lib/LTO/LTO.cpp</div><div>index 7e8b9b3c6390..ee11d07d6b8e 100644</div><div>--- a/llvm/lib/LTO/LTO.cpp</div><div>+++ b/llvm/lib/LTO/LTO.cpp</div><div>@@ -287,7 +287,7 @@ static void thinLTOResolveWeakForLinkerGUID(</div><div> recordNewLinkage) {</div><div> for (auto &S : GVSummaryList) {</div><div> GlobalValue::LinkageTypes OriginalLinkage = S->linkage();</div><div>- if (!GlobalValue::isWeakForLinker(OriginalLinkage))</div><div>+ if (GlobalValue::isLocalLinkage(OriginalLinkage))</div><div> continue;</div><div> // We need to emit only one of these. The prevailing module will keep it,</div><div> // but turned into a weak, while the others will drop it when possible.</div><div>diff --git a/llvm/lib/Transforms/IPO/FunctionImport.cpp b/llvm/lib/Transforms/IPO/FunctionImport.cpp</div><div>index 246d75caefa2..61790c9fc435 100644</div><div>--- a/llvm/lib/Transforms/IPO/FunctionImport.cpp</div><div>+++ b/llvm/lib/Transforms/IPO/FunctionImport.cpp</div><div>@@ -765,7 +765,7 @@ void llvm::thinLTOResolveWeakForLinkerModule(</div><div> return;</div><div> }</div><div> </div><div>- if (!GlobalValue::isWeakForLinker(GV.getLinkage()))</div><div>+ if (GlobalValue::isLocalLinkage(GV.getLinkage()))</div><div> return;</div><div> // Check for a non-prevailing def that has interposable linkage</div><div> // (e.g. non-odr weak or linkonce). In that case we can't simply</div></div><div><br></div><div>Peter</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 9, 2018 at 10:06 AM, Teresa Johnson <span dir="ltr"><<a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Adding Peter to comment on the linker resolution issue.<br><div><br></div><div>From adding save-temps, it looks like lld and gold are giving different resolutions to the symbols, which is presumably creating this issue:</div><div><br></div><div>(first file is with lld and second is with gold)</div><div><br></div><div><div>$ diff a.out.resolution.txt gold/</div><div>4c4</div><div>< -r=a.o,__llvm_profile_raw_version,plx</div><div>---</div><div>> -r=a.o,__llvm_profile_raw_version,l</div><div>8,9c8,9</div><div>< -r=b.o,__llvm_profile_raw_version,x</div><div>< -r=b.o,__llvm_profile_filename,x</div><div>---</div><div>> -r=b.o,__llvm_profile_raw_version,l</div><div>> -r=b.o,__llvm_profile_filename,l</div></div><div><br></div><div>I.e. with lld they are all marked as exported.</div><div><br></div><div>Peter, any ideas?</div><span class="m_3031369597707563578HOEnZb"><font color="#888888"><div><br></div><div>Teresa</div></font></span></div><div class="m_3031369597707563578HOEnZb"><div class="m_3031369597707563578h5"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr">On Wed, May 9, 2018 at 9:52 AM Teresa Johnson <<a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Sorry, operator error. I can reproduce now. Interestingly, this does not reproduce using gold, and they utilize the same underlying LTO API. Let me dig a little using save-temps and see where they diverge.<br><div>Teresa</div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, May 9, 2018 at 9:28 AM Pirama Arumuga Nainar <<a href="mailto:pirama@google.com" target="_blank">pirama@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">LLD revision is r331862. To add, I had initially tried it on r328903, which also reproduced the issue.</div><br><div class="gmail_quote"><div dir="ltr">On Wed, May 9, 2018 at 9:26 AM Pirama Arumuga Nainar <<a href="mailto:pirama@google.com" target="_blank">pirama@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Teresa,<div><br></div><div>Thanks for looking into this. I hadn't initially tried ToT, but it reproduces in ToT as well when I tried.</div><div><br></div><div><pre id="m_3031369597707563578m_8378737717857792049m_-1512482089208273594m_4837983272749416503m_-6213466112148886259m_5408140560637564606gmail-hterm:copy-to-clipboard-source" style="font-size:13px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;color:rgb(0,0,0)">$ ./clang --version
</pre><pre id="m_3031369597707563578m_8378737717857792049m_-1512482089208273594m_4837983272749416503m_-6213466112148886259m_5408140560637564606gmail-hterm:copy-to-clipboard-source" style="color:rgb(0,0,0);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">clang version 7.0.0 (trunk 331879) (llvm/trunk 331888)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /ssd2/pirama/llvm-upstream/llvm-release/bin/.</pre><pre id="m_3031369597707563578m_8378737717857792049m_-1512482089208273594m_4837983272749416503m_-6213466112148886259m_5408140560637564606gmail-hterm:copy-to-clipboard-source" style="color:rgb(0,0,0);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">$ ./ld.lld --version
LLD 7.0.0 (<a href="https://git.llvm.org/git/lld.git" target="_blank">https://git.llvm.org/git/lld.git</a> dc4977e104c7f33758ac6119d1528aafcd8a94dc) (compatible with GNU linkers)</pre></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, May 9, 2018 at 6:44 AM Teresa Johnson <<a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Wed, May 9, 2018 at 6:43 AM Teresa Johnson <<a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Pirama,<br><div><br></div><div>I can't reproduce with either lld or gold, using a compiler built from head. What version is your clang?</div></div></blockquote><div>(and your lld)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>Thanks,</div><div>Teresa</div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, May 8, 2018 at 7:50 PM Pirama Arumuga Nainar via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The duplicate symbol errors are for <span style="color:rgb(34,34,34);font-family:sans-serif;font-size:13px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">__llvm_profile_filename and <span style="color:rgb(34,34,34);font-family:sans-serif;font-size:13px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">__llvm_profile_raw_version. IIUC, these are supposed to be weak symbols but Thin LTO seems to break this in some way. </span></span>This does't happen with gold or no LTO or full LTO.<div><br></div><div>$ cat > a.c</div><div><div>extern int foo();</div><div><br></div><div>int main() {</div><div> return foo();</div><div>}</div></div><div><br></div><div>$ cat > b.c</div><div><div>int foo() {</div><div> return 0;</div><div>}</div></div><div><br>$ clang a.c -fprofile-generate -flto=thin -c</div><div>$ clang b.c -fprofile-generate -flto=thin -c</div><div>$ clang a.o b.o -fprofile-generate -flto=thin -fuse-ld=lld</div><div>ld.lld: error: duplicate symbol: __llvm_profile_filename</div><div>>>> defined at a.c</div><div>>>> lto.tmp:(__llvm_profile_filename)</div><div>>>> defined at b.c</div><div>>>> lto.tmp:(.rodata.__llvm_profile_filename+0x0)</div><div><br></div><div>ld.lld: error: duplicate symbol: __llvm_profile_raw_version</div><div>>>> defined at a.c</div><div>>>> lto.tmp:(__llvm_profile_raw_version)</div><div>>>> defined at b.c</div><div>>>> lto.tmp:(.rodata.__llvm_profile_raw_version+0x0)</div><div>clang-7: error: linker command failed with exit code 1 (use -v to see invocation)</div><div>Makefile:10: recipe for target 'a.out' failed</div><div>make: *** [a.out] Error 1</div></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_3031369597707563578m_8378737717857792049m_-1512482089208273594m_4837983272749416503m_-6213466112148886259m_5408140560637564606m_6765173187228408672m_-1099755374332112270gmail_signature" data-smartmail="gmail_signature"><span style="font-family:Times;font-size:medium"><table cellspacing="0" cellpadding="0"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small"><td style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Teresa Johnson |</td><td style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td><td style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a> |</td><td style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"> 408-460-2413</td></tr></tbody></table></span></div></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_3031369597707563578m_8378737717857792049m_-1512482089208273594m_4837983272749416503m_-6213466112148886259m_5408140560637564606m_6765173187228408672gmail_signature" data-smartmail="gmail_signature"><span style="font-family:Times;font-size:medium"><table cellspacing="0" cellpadding="0"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small"><td style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Teresa Johnson |</td><td style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td><td style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a> |</td><td style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"> 408-460-2413</td></tr></tbody></table></span></div></div>
</blockquote></div></blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_3031369597707563578m_8378737717857792049m_-1512482089208273594m_4837983272749416503gmail_signature" data-smartmail="gmail_signature"><span style="font-family:Times;font-size:medium"><table cellspacing="0" cellpadding="0"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small"><td style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Teresa Johnson |</td><td style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td><td style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a> |</td><td style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"> 408-460-2413</td></tr></tbody></table></span></div></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_3031369597707563578m_8378737717857792049m_-1512482089208273594gmail_signature" data-smartmail="gmail_signature"><span style="font-family:Times;font-size:medium"><table cellspacing="0" cellpadding="0"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small"><td style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Teresa Johnson |</td><td style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td><td style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a> |</td><td style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"> 408-460-2413</td></tr></tbody></table></span></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_3031369597707563578gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">-- <div>Peter</div></div></div>
</div>
</blockquote></div>