<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">This test flaked here:<div><a href="http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/13465/">http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/13465/</a><br></div><div><br></div><div>The wasm equivalent one also failed three builds later:</div><div><a href="http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/13468">http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/13468</a></div><div><br></div><div>Perhaps we should disable the cache policy test until these issues are resolved?</div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Oct 17, 2018 at 9:38 AM Chandler Carruth via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I don't have more recent bot failures handy, but you can reproduce this by running the test repeatedly I suspect....<div><br></div><div>That said, maybe the patch you mention will address it?</div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Oct 15, 2018 at 9:03 AM Teresa Johnson <<a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr">Do you have a more recent bot failure you could point at? The previous one has expired. I wonder if this is:<div><a href="https://reviews.llvm.org/D53123" target="_blank">https://reviews.llvm.org/D53123</a> (was fixed for the equivalent gold test recently, and I asked that they fix for lld as well. I just found that they indeed sent a patch to fix that one too last week, and I just LGTMed).<br></div><div><br></div><div>Teresa</div></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Oct 15, 2018 at 2:06 AM Chandler Carruth via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Ping!<div><br></div><div>I continue to see spurious failures of the LTO cache tests for LLD on every architecture that supports it. This is a really frustrating thing to have just sit in tree and constant break bots and developers.</div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Sep 4, 2018 at 5:58 AM Chandler Carruth <<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Sorry to dig up an old commit, but...<br><br><div class="gmail_quote"></div></div><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Thu, Mar 2, 2017 at 12:11 AM Peter Collingbourne via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: pcc<br>
Date: Wed Mar 1 17:00:10 2017<br>
New Revision: 296702<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=296702&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=296702&view=rev</a><br>
Log:<br>
ELF: Add ThinLTO caching support.<br>
<br>
This patch adds an option named --thinlto-cache-dir, which specifies the<br>
path to a directory in which to cache native object files for ThinLTO<br>
incremental builds.<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D30509" rel="noreferrer" target="_blank">https://reviews.llvm.org/D30509</a><br>
<br>
Added:<br>
lld/trunk/test/ELF/lto/Inputs/cache.ll<br>
lld/trunk/test/ELF/lto/cache.ll<br></blockquote><div><br></div></div></div><div dir="ltr"><div class="gmail_quote"><div>I have found this test to be frustratingly flaky. Most recent example:</div><div><a href="http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/12451" target="_blank">http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/12451</a><br></div><div><br></div><div>It is particularly interesting that it is flaky on this linux bot.</div><div><br></div><div>Anything else we can do here?</div></div></div><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Modified:<br>
lld/trunk/ELF/Config.h<br>
lld/trunk/ELF/Driver.cpp<br>
lld/trunk/ELF/LTO.cpp<br>
lld/trunk/ELF/LTO.h<br>
lld/trunk/ELF/Options.td<br>
<br>
Modified: lld/trunk/ELF/Config.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Config.h?rev=296702&r1=296701&r2=296702&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Config.h?rev=296702&r1=296701&r2=296702&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/ELF/Config.h (original)<br>
+++ lld/trunk/ELF/Config.h Wed Mar 1 17:00:10 2017<br>
@@ -86,6 +86,7 @@ struct Configuration {<br>
llvm::StringRef OptRemarksFilename;<br>
llvm::StringRef SoName;<br>
llvm::StringRef Sysroot;<br>
+ llvm::StringRef ThinLTOCacheDir;<br>
std::string RPath;<br>
std::vector<VersionDefinition> VersionDefinitions;<br>
std::vector<llvm::StringRef> AuxiliaryList;<br>
<br>
Modified: lld/trunk/ELF/Driver.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Driver.cpp?rev=296702&r1=296701&r2=296702&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Driver.cpp?rev=296702&r1=296701&r2=296702&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/ELF/Driver.cpp (original)<br>
+++ lld/trunk/ELF/Driver.cpp Wed Mar 1 17:00:10 2017<br>
@@ -583,6 +583,7 @@ void LinkerDriver::readConfigs(opt::Inpu<br>
Config->Sysroot = getString(Args, OPT_sysroot);<br>
Config->Target1Rel = getArg(Args, OPT_target1_rel, OPT_target1_abs, false);<br>
Config->Target2 = getTarget2(Args);<br>
+ Config->ThinLTOCacheDir = getString(Args, OPT_thinlto_cache_dir);<br>
Config->ThinLTOJobs = getInteger(Args, OPT_thinlto_jobs, -1u);<br>
Config->Threads = getArg(Args, OPT_threads, OPT_no_threads, true);<br>
Config->Trace = Args.hasArg(OPT_trace);<br>
<br>
Modified: lld/trunk/ELF/LTO.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/LTO.cpp?rev=296702&r1=296701&r2=296702&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/LTO.cpp?rev=296702&r1=296701&r2=296702&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/ELF/LTO.cpp (original)<br>
+++ lld/trunk/ELF/LTO.cpp Wed Mar 1 17:00:10 2017<br>
@@ -18,6 +18,7 @@<br>
#include "llvm/ADT/StringRef.h"<br>
#include "llvm/ADT/Twine.h"<br>
#include "llvm/IR/DiagnosticPrinter.h"<br>
+#include "llvm/LTO/Caching.h"<br>
#include "llvm/LTO/Config.h"<br>
#include "llvm/LTO/LTO.h"<br>
#include "llvm/Object/SymbolicFile.h"<br>
@@ -142,11 +143,24 @@ std::vector<InputFile *> BitcodeCompiler<br>
std::vector<InputFile *> Ret;<br>
unsigned MaxTasks = LTOObj->getMaxTasks();<br>
Buff.resize(MaxTasks);<br>
+ Files.resize(MaxTasks);<br>
<br>
- checkError(LTOObj->run([&](size_t Task) {<br>
- return llvm::make_unique<lto::NativeObjectStream>(<br>
- llvm::make_unique<raw_svector_ostream>(Buff[Task]));<br>
- }));<br>
+ // The --thinlto-cache-dir option specifies the path to a directory in which<br>
+ // to cache native object files for ThinLTO incremental builds. If a path was<br>
+ // specified, configure LTO to use it as the cache directory.<br>
+ lto::NativeObjectCache Cache;<br>
+ if (!Config->ThinLTOCacheDir.empty())<br>
+ Cache = lto::localCache(Config->ThinLTOCacheDir,<br>
+ [&](size_t Task, StringRef Path) {<br>
+ Files[Task] = check(MemoryBuffer::getFile(Path));<br>
+ });<br>
+<br>
+ checkError(LTOObj->run(<br>
+ [&](size_t Task) {<br>
+ return llvm::make_unique<lto::NativeObjectStream>(<br>
+ llvm::make_unique<raw_svector_ostream>(Buff[Task]));<br>
+ },<br>
+ Cache));<br>
<br>
for (unsigned I = 0; I != MaxTasks; ++I) {<br>
if (Buff[I].empty())<br>
@@ -160,5 +174,10 @@ std::vector<InputFile *> BitcodeCompiler<br>
InputFile *Obj = createObjectFile(MemoryBufferRef(Buff[I], "lto.tmp"));<br>
Ret.push_back(Obj);<br>
}<br>
+<br>
+ for (std::unique_ptr<MemoryBuffer> &File : Files)<br>
+ if (File)<br>
+ Ret.push_back(createObjectFile(*File));<br>
+<br>
return Ret;<br>
}<br>
<br>
Modified: lld/trunk/ELF/LTO.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/LTO.h?rev=296702&r1=296701&r2=296702&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/LTO.h?rev=296702&r1=296701&r2=296702&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/ELF/LTO.h (original)<br>
+++ lld/trunk/ELF/LTO.h Wed Mar 1 17:00:10 2017<br>
@@ -49,6 +49,7 @@ public:<br>
private:<br>
std::unique_ptr<llvm::lto::LTO> LTOObj;<br>
std::vector<SmallString<0>> Buff;<br>
+ std::vector<std::unique_ptr<MemoryBuffer>> Files;<br>
};<br>
}<br>
}<br>
<br>
Modified: lld/trunk/ELF/Options.td<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Options.td?rev=296702&r1=296701&r2=296702&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Options.td?rev=296702&r1=296701&r2=296702&view=diff</a><br>
==============================================================================<br>
--- lld/trunk/ELF/Options.td (original)<br>
+++ lld/trunk/ELF/Options.td Wed Mar 1 17:00:10 2017<br>
@@ -392,4 +392,6 @@ def opt_remarks_filename: S<"opt-remarks<br>
def opt_remarks_with_hotness: F<"opt-remarks-with-hotness">,<br>
HelpText<"Include hotness informations in the optimization remarks file">;<br>
def save_temps: F<"save-temps">;<br>
+def thinlto_cache_dir: J<"thinlto-cache-dir=">,<br>
+ HelpText<"Path to ThinLTO cached object file directory">;<br>
def thinlto_jobs: J<"thinlto-jobs=">, HelpText<"Number of ThinLTO jobs">;<br>
<br>
Added: lld/trunk/test/ELF/lto/Inputs/cache.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/lto/Inputs/cache.ll?rev=296702&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/lto/Inputs/cache.ll?rev=296702&view=auto</a><br>
==============================================================================<br>
--- lld/trunk/test/ELF/lto/Inputs/cache.ll (added)<br>
+++ lld/trunk/test/ELF/lto/Inputs/cache.ll Wed Mar 1 17:00:10 2017<br>
@@ -0,0 +1,10 @@<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"<br>
+target triple = "x86_64-unknown-linux-gnu"<br>
+<br>
+define i32 @main() {<br>
+entry:<br>
+ call void (...) @globalfunc()<br>
+ ret i32 0<br>
+}<br>
+<br>
+declare void @globalfunc(...)<br>
<br>
Added: lld/trunk/test/ELF/lto/cache.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/lto/cache.ll?rev=296702&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/lto/cache.ll?rev=296702&view=auto</a><br>
==============================================================================<br>
--- lld/trunk/test/ELF/lto/cache.ll (added)<br>
+++ lld/trunk/test/ELF/lto/cache.ll Wed Mar 1 17:00:10 2017<br>
@@ -0,0 +1,15 @@<br>
+; RUN: opt -module-hash -module-summary %s -o %t.o<br>
+; RUN: opt -module-hash -module-summary %p/Inputs/cache.ll -o %t2.o<br>
+<br>
+; RUN: rm -Rf %t.cache && mkdir %t.cache<br>
+; RUN: ld.lld --thinlto-cache-dir=%t.cache -o %t3.o %t2.o %t.o<br>
+<br>
+; RUN: ls %t.cache | count 2<br>
+<br>
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"<br>
+target triple = "x86_64-unknown-linux-gnu"<br>
+<br>
+define void @globalfunc() #0 {<br>
+entry:<br>
+ ret void<br>
+}<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div></div></blockquote></div>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_3858204926423369584m_-5237226397381066260gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><span style="font-family:Times;font-size:medium"><table cellspacing="0" cellpadding="0"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small"><td nowrap style="border-top:2px solid rgb(213,15,37)">Teresa Johnson |</td><td nowrap style="border-top:2px solid rgb(51,105,232)"> Software Engineer |</td><td nowrap style="border-top:2px solid rgb(0,153,57)"> <a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a> |</td><td nowrap style="border-top:2px solid rgb(238,178,17)"><br></td></tr></tbody></table></span></div></div></div>
</blockquote></div>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div>