<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Oct 8, 2016 at 4:00 PM, Mehdi Amini <span dir="ltr"><<a href="mailto:mehdi.amini@apple.com" target="_blank">mehdi.amini@apple.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Hal!<br>
<span><br>
<br>
> On Oct 8, 2016, at 3:51 PM, Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>> wrote:<br>
><br>
> Hi Mehdi,<br>
><br>
> test/ThinLTO/X86/cache.ll now fails on my build system, which uses:<br>
<br>
</span>Looking at the trace below, it looks like test/tools/gold/X86/cache.ll<br>
<span><br>
><br>
> $ /usr/bin/ld.gold --version<br>
> GNU gold (version 2.23.52.0.1-55.el7 20130226) 1.11<br>
><br>
> And runs these commands:<br>
><br>
> /path/to/build/llvm-stage1/./b<wbr>in/opt -module-summary /path/to/src/llvm/test/tools/g<wbr>old/X86/cache.ll -o /path/to/build/llvm-stage1/tes<wbr>t/tools/gold/X86/Output/cache.<wbr>ll.tmp.o<br>
> /path/to/build/llvm-stage1/./b<wbr>in/opt -module-summary /path/to/src/llvm/test/tools/g<wbr>old/X86/Inputs/cache.ll -o /path/to/build/llvm-stage1/tes<wbr>t/tools/gold/X86/Output/cache.<wbr>ll.tmp2.o<br>
> rm -Rf /path/to/build/llvm-stage1/tes<wbr>t/tools/gold/X86/Output/cache.<wbr>ll.tmp.cache && mkdir /path/to/build/llvm-stage1/tes<wbr>t/tools/gold/X86/Output/cache.<wbr>ll.tmp.cache<br>
> /usr/bin/ld.gold -m elf_x86_64 -plugin /path/to/build/llvm-stage1/./l<wbr>ib/LLVMgold.so --plugin-opt=thinlto --plugin-opt=cache-dir=/path/t<wbr>o/build/llvm-stage1/test/tools<wbr>/gold/X86/Output/cache.ll.tmp.<wbr>cache -o /path/to/build/llvm-stage1/tes<wbr>t/tools/gold/X86/Output/cache.<wbr>ll.tmp3.o /path/to/build/llvm-stage1/tes<wbr>t/tools/gold/X86/Output/cache.<wbr>ll.tmp2.o /path/to/build/llvm-stage1/tes<wbr>t/tools/gold/X86/Output/cache.<wbr>ll.tmp.o<br>
> ls /path/to/build/llvm-stage1/tes<wbr>t/tools/gold/X86/Output/cache.<wbr>ll.tmp.cache | /path/to/build/llvm-stage1/./b<wbr>in/count 2<br>
><br>
> All of the commands except the final one, specifically including the ld.gold command, succeed. The Output/cache.ll.tmp3.o output is indeed created. However, the cache.ll.tmp.cache is empty, and so the final count check fails.<br>
<br>
</span>Makes sense, this is the same test as test/ThinLTO/X86/cache.ll ; I updated it the same way in r283681.<br>
Let me know if it fixes it for you!<br>
<br>
+CC Teresa: Don’t we have a bot that runs with Gold?<br></blockquote><div><br></div><div>Yes, almost certainly there are several. I haven't set up any bots myself, but a quick look at the git log of test/tools/gold/ shows that I have made several test fixes due to bot failures. Some of the bots are listed in those commit messages.</div><div><br></div><div>Maybe the bots were already failing due to something else and so you didn't get any notifications?</div><div><br></div><div>Teresa</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
—<br>
<span class="m_-8486260203418921679HOEnZb"><font color="#888888">Mehdi<br>
</font></span><div class="m_-8486260203418921679HOEnZb"><div class="m_-8486260203418921679h5"><br>
<br>
<br>
><br>
> -Hal<br>
><br>
> ----- Original Message -----<br>
>> From: "Mehdi Amini via llvm-commits" <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>><br>
>> To: <a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
>> Sent: Friday, October 7, 2016 11:44:24 PM<br>
>> Subject: [llvm] r283655 - ThinLTO: don't perform incremental LTO on module without a hash<br>
>><br>
>> Author: mehdi_amini<br>
>> Date: Fri Oct 7 23:44:23 2016<br>
>> New Revision: 283655<br>
>><br>
>> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=283655&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject?rev=283655&view=rev</a><br>
>> Log:<br>
>> ThinLTO: don't perform incremental LTO on module without a hash<br>
>><br>
>> Clang always emit a hash for ThinLTO, but as other frontend are<br>
>> starting to use ThinLTO, this could be a serious bug.<br>
>><br>
>> Differential Revision: <a href="https://reviews.llvm.org/D25379" rel="noreferrer" target="_blank">https://reviews.llvm.org/D2537<wbr>9</a><br>
>><br>
>> Modified:<br>
>> llvm/trunk/lib/LTO/LTO.cpp<br>
>> llvm/trunk/lib/LTO/ThinLTOCode<wbr>Generator.cpp<br>
>> llvm/trunk/test/ThinLTO/X86/ca<wbr>che.ll<br>
>> llvm/trunk/test/ThinLTO/X86/em<wbr>pty_module_with_cache.ll<br>
>><br>
>> Modified: llvm/trunk/lib/LTO/LTO.cpp<br>
>> URL:<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/LTO.cpp?rev=283655&r1=283654&r2=283655&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/llvm/trunk/lib/LTO/LTO.<wbr>cpp?rev=283655&r1=283654&r2=<wbr>283655&view=diff</a><br>
>> ==============================<wbr>==============================<wbr>==================<br>
>> --- llvm/trunk/lib/LTO/LTO.cpp (original)<br>
>> +++ llvm/trunk/lib/LTO/LTO.cpp Fri Oct 7 23:44:23 2016<br>
>> @@ -542,8 +542,12 @@ public:<br>
>> };<br>
>><br>
>> auto ModuleID = MBRef.getBufferIdentifier();<br>
>> - if (!Cache || !CombinedIndex.modulePaths().c<wbr>ount(ModuleID))<br>
>> - // Cache disabled or no entry for this module in the combined<br>
>> index<br>
>> +<br>
>> + if (!Cache || !CombinedIndex.modulePaths().c<wbr>ount(ModuleID) ||<br>
>> + all_of(CombinedIndex.getModule<wbr>Hash(ModuleID),<br>
>> + [](uint32_t V) { return V == 0; }))<br>
>> + // Cache disabled or no entry for this module in the combined<br>
>> index or<br>
>> + // no module hash.<br>
>> return RunThinBackend(AddStream);<br>
>><br>
>> SmallString<40> Key;<br>
>><br>
>> Modified: llvm/trunk/lib/LTO/ThinLTOCode<wbr>Generator.cpp<br>
>> URL:<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp?rev=283655&r1=283654&r2=283655&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/llvm/trunk/lib/LTO/ThinL<wbr>TOCodeGenerator.cpp?rev=283655<wbr>&r1=283654&r2=283655&view=diff</a><br>
>> ==============================<wbr>==============================<wbr>==================<br>
>> --- llvm/trunk/lib/LTO/ThinLTOCode<wbr>Generator.cpp (original)<br>
>> +++ llvm/trunk/lib/LTO/ThinLTOCode<wbr>Generator.cpp Fri Oct 7 23:44:23<br>
>> 2016<br>
>> @@ -243,6 +243,13 @@ public:<br>
>> // export list, the hash for every single module in the import<br>
>> list, the<br>
>> // list of ResolvedODR for the module, and the list of preserved<br>
>> symbols.<br>
>><br>
>> + // Include the hash for the current module<br>
>> + auto ModHash = Index.getModuleHash(ModuleID);<br>
>> +<br>
>> + if (all_of(ModHash, [](uint32_t V) { return V == 0; }))<br>
>> + // No hash entry, no caching!<br>
>> + return;<br>
>> +<br>
>> SHA1 Hasher;<br>
>><br>
>> // Start with the compiler revision<br>
>> @@ -251,8 +258,6 @@ public:<br>
>> Hasher.update(LLVM_REVISION);<br>
>> #endif<br>
>><br>
>> - // Include the hash for the current module<br>
>> - auto ModHash = Index.getModuleHash(ModuleID);<br>
>> Hasher.update(ArrayRef<uint8_<wbr>t>((uint8_t *)&ModHash[0],<br>
>> sizeof(ModHash)));<br>
>> for (auto F : ExportList)<br>
>> // The export list can impact the internalization, be<br>
>> conservative here<br>
>><br>
>> Modified: llvm/trunk/test/ThinLTO/X86/ca<wbr>che.ll<br>
>> URL:<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ThinLTO/X86/cache.ll?rev=283655&r1=283654&r2=283655&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/llvm/trunk/test/ThinLTO/<wbr>X86/cache.ll?rev=283655&r1=<wbr>283654&r2=283655&view=diff</a><br>
>> ==============================<wbr>==============================<wbr>==================<br>
>> --- llvm/trunk/test/ThinLTO/X86/ca<wbr>che.ll (original)<br>
>> +++ llvm/trunk/test/ThinLTO/X86/ca<wbr>che.ll Fri Oct 7 23:44:23 2016<br>
>> @@ -1,6 +1,28 @@<br>
>> +; Verify first that *without* hash, we don't use the cache.<br>
>> +<br>
>> ; RUN: opt -module-summary %s -o %t.bc<br>
>> ; RUN: opt -module-summary %p/Inputs/cache.ll -o %t2.bc<br>
>><br>
>> +; Verify that enabling caching is ignoring module without hash<br>
>> +; RUN: rm -Rf %t.cache && mkdir %t.cache<br>
>> +; RUN: llvm-lto -thinlto-action=run -exported-symbol=globalfunc<br>
>> %t2.bc %t.bc -thinlto-cache-dir %t.cache<br>
>> +; RUN: ls %t.cache/llvmcache.timestamp<br>
>> +; RUN: ls %t.cache | count 1<br>
>> +<br>
>> +; Verify that enabling caching is ignoring module without hash with<br>
>> llvm-lto2<br>
>> +; RUN: rm -Rf %t.cache && mkdir %t.cache<br>
>> +; RUN: llvm-lto2 -o %t.o %t2.bc %t.bc -cache-dir %t.cache \<br>
>> +; RUN: -r=%t2.bc,_main,plx \<br>
>> +; RUN: -r=%t2.bc,_globalfunc,lx \<br>
>> +; RUN: -r=%t.bc,_globalfunc,plx<br>
>> +; RUN: ls %t.cache | count 0<br>
>> +<br>
>> +<br>
>> +; Repeat again, *with* hash this time.<br>
>> +<br>
>> +; RUN: opt -module-hash -module-summary %s -o %t.bc<br>
>> +; RUN: opt -module-hash -module-summary %p/Inputs/cache.ll -o %t2.bc<br>
>> +<br>
>> ; Verify that enabling caching is working<br>
>> ; RUN: rm -Rf %t.cache && mkdir %t.cache<br>
>> ; RUN: llvm-lto -thinlto-action=run -exported-symbol=globalfunc<br>
>> %t2.bc %t.bc -thinlto-cache-dir %t.cache<br>
>><br>
>> Modified: llvm/trunk/test/ThinLTO/X86/em<wbr>pty_module_with_cache.ll<br>
>> URL:<br>
>> <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ThinLTO/X86/empty_module_with_cache.ll?rev=283655&r1=283654&r2=283655&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/llvm/trunk/test/ThinLTO/<wbr>X86/empty_module_with_cache.<wbr>ll?rev=283655&r1=283654&r2=<wbr>283655&view=diff</a><br>
>> ==============================<wbr>==============================<wbr>==================<br>
>> --- llvm/trunk/test/ThinLTO/X86/em<wbr>pty_module_with_cache.ll (original)<br>
>> +++ llvm/trunk/test/ThinLTO/X86/em<wbr>pty_module_with_cache.ll Fri Oct 7<br>
>> 23:44:23 2016<br>
>> @@ -22,13 +22,13 @@<br>
>> ; RUN: rm -Rf %t.cache && mkdir %t.cache<br>
>> ; RUN: llvm-lto -thinlto-action=run %t2.bc %t.bc<br>
>> -exported-symbol=main -thinlto-cache-dir %t.cache<br>
>> ; RUN: ls %t.cache/llvmcache.timestamp<br>
>> -; RUN: ls %t.cache | count 2<br>
>> +; RUN: ls %t.cache | count 1<br>
>><br>
>> ; Verify that caching is disabled for module without hash, with<br>
>> llvm-lto2<br>
>> ; RUN: rm -Rf %t.cache && mkdir %t.cache<br>
>> ; RUN: llvm-lto2 -o %t.o %t2.bc %t.bc -cache-dir %t.cache \<br>
>> ; RUN: -r=%t2.bc,_main,plx<br>
>> -; RUN: ls %t.cache | count 1<br>
>> +; RUN: ls %t.cache | count 0<br>
>><br>
>><br>
>> target datalayout = "e-m:o-i64:64-f80:128-n8:16:32<wbr>:64-S128"<br>
>><br>
>><br>
>> ______________________________<wbr>_________________<br>
>> llvm-commits mailing list<br>
>> <a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br>
>><br>
><br>
> --<br>
> Hal Finkel<br>
> Lead, Compiler Technology and Programming Languages<br>
> Leadership Computing Facility<br>
> Argonne National Laboratory<br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_-8486260203418921679gmail_signature" data-smartmail="gmail_signature"><span style="font-family:Times;font-size:medium"><table cellspacing="0" cellpadding="0"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small"><td nowrap style="border-top-style:solid;border-top-color:rgb(213,15,37);border-top-width:2px">Teresa Johnson |</td><td nowrap style="border-top-style:solid;border-top-color:rgb(51,105,232);border-top-width:2px"> Software Engineer |</td><td nowrap style="border-top-style:solid;border-top-color:rgb(0,153,57);border-top-width:2px"> <a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a> |</td><td nowrap style="border-top-style:solid;border-top-color:rgb(238,178,17);border-top-width:2px"> <a href="tel:408-460-2413" value="+14084602413" target="_blank">408-460-2413</a></td></tr></tbody></table></span></div>
</div></div>