<div dir="ltr">MSan buildbot crashes are still happening, once per 8 builds or so. I can't reproduce them locally and am not 100% sure that this is the change, but, given the nondeterminism detected on the other bot, I'm going to tentatively revert it.<div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 12, 2018 at 12:01 PM, Galina Kistanova <span dir="ltr"><<a href="mailto:gkistanova@gmail.com" target="_blank">gkistanova@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hello Tim,</div><div><br></div><div>This commit has introduced nondeterminism. Please see <a href="http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu" target="_blank">http://lab.llvm.org:8011/<wbr>builders/clang-with-thin-lto-<wbr>ubuntu</a> or <a href="http://lab.llvm.org:8011/builders/clang-with-lto-ubuntu" target="_blank">http://lab.llvm.org:8011/<wbr>builders/clang-with-lto-ubuntu</a> build bots.</div><div><br></div><div>This is in addition to those other problems mentioned already, or maybe some of the other problems are exposed because of this.<br></div><div><br></div><div>Since the problem is around for quite long now, maybe it worth reverting till you will figure out what's wrong and how to fix it?</div><div><br></div><div>Thanks</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Galina<br></div><div><br></div><div><br></div><div><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 10, 2018 at 1:33 PM, Evgenii Stepanov via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi Tim,</div><div><br></div><div>MSan bootstrap bot has been randomly crashing lately with the following trace:</div><div><div><br></div><div>3.<span style="white-space:pre-wrap"> </span>Running pass 'CallGraph Pass Manager' on module '/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm/lib/Tar<wbr>get/Hexagon/HexagonCommonGEP.<wbr>cpp'.</div><div>#0 0x00005602f79f5cfa llvm::sys::PrintStackTrace(llv<wbr>m::raw_ostream&) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x24f5cfa)</div><div>#1 0x00005602f79f42c5 llvm::sys::RunSignalHandlers() (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x24f42c5)</div><div>#2 0x00005602f79f43dc SignalHandler(int) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x24f43dc)</div><div>#3 0x00007f63daa840c0 __restore_rt (/lib/x86_64-linux-gnu/libpthr<wbr>ead.so.0+0x110c0)</div><div>#4 0x00005602f6edf38e llvm::scc_iterator<llvm::CallG<wbr>raph*, llvm::GraphTraits<llvm::CallGr<wbr>aph*> >::DFSVisitChildren() (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x19df38e)</div><div>#5 0x00005602f6edf5aa llvm::scc_iterator<llvm::CallG<wbr>raph*, llvm::GraphTraits<llvm::CallGr<wbr>aph*> >::GetNextSCC() (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x19df5aa)</div><div>#6 0x00005602f6ee05ed (anonymous namespace)::CGPassManager::run<wbr>OnModule(llvm::Module&) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x19e05ed)</div><div>#7 0x00005602f74ca5fd llvm::legacy::PassManagerImpl:<wbr>:run(llvm::Module&) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x1fca5fd)</div><div>#8 0x00005602f7bde5c1 (anonymous namespace)::EmitAssemblyHelper<wbr>::EmitAssembly(clang::BackendA<wbr>ction, std::unique_ptr<llvm::raw_pwri<wbr>te_stream, std::default_delete<llvm::raw_<wbr>pwrite_stream> >) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x26de5c1)</div><div>#9 0x00005602f7bdfdf0 clang::EmitBackendOutput(clang<wbr>::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwri<wbr>te_stream, std::default_delete<llvm::raw_<wbr>pwrite_stream> >) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x26dfdf0)</div><div>#10 0x00005602f836f264 clang::BackendConsumer::Handle<wbr>TranslationUnit(clang::ASTCont<wbr>ext&) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x2e6f264)</div><div>#11 0x00005602f8b37fa9 clang::ParseAST(clang::Sema&, bool, bool) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x3637fa9)</div><div>#12 0x00005602f836e4d9 clang::CodeGenAction::ExecuteA<wbr>ction() (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x2e6e4d9)</div><div>#13 0x00005602f800da7e clang::FrontendAction::Execute<wbr>() (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x2b0da7e)</div><div>#14 0x00005602f7fd9496 clang::CompilerInstance::Execu<wbr>teAction(clang::FrontendAction<wbr>&) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x2ad9496)</div><div>#15 0x00005602f80ad511 clang::ExecuteCompilerInvocati<wbr>on(clang::CompilerInstance*) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0x2bad511)</div><div>#16 0x00005602f614c080 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0xc4c080)</div><div>#17 0x00005602f60c1a22 main (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0xbc1a22)</div><div>#18 0x00007f63d98312e1 __libc_start_main (/lib/x86_64-linux-gnu/libc.so<wbr>.6+0x202e1)</div><div>#19 0x00005602f61480aa _start (/b/sanitizer-x86_64-linux-boo<wbr>tstrap-msan/build/llvm_build0/<wbr>bin/clang-7+0xc480aa)</div><div>clang-7: error: unable to execute command: Segmentation fault</div></div><div><br></div><div>The crash happens roughly in 1 out of 5-to-10 builds, see history here:</div><a href="http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-msan?numbuilds=300" target="_blank">http://lab.llvm.org:8011/build<wbr>ers/sanitizer-x86_64-linux-<wbr>bootstrap-msan?numbuilds=300</a><br><div><br></div><div>The earliest it happened was this build at r336432:</div><div><a href="http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-msan/builds/5763" target="_blank">http://lab.llvm.org:8011/build<wbr>ers/sanitizer-x86_64-linux-<wbr>bootstrap-msan/builds/5763</a><br></div><div><br></div><div>I suspect that this change could be the culprit. Could you take a look?</div><div><br></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jul 6, 2018 at 1:04 AM, Tim Northover via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: tnorthover<br>
Date: Fri Jul 6 01:04:47 2018<br>
New Revision: 336419<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=336419&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject?rev=336419&view=rev</a><br>
Log:<br>
CallGraphSCCPass: iterate over all functions.<br>
<br>
Previously we only iterated over functions reachable from the set of<br>
external functions in the module. But since some of the passes under<br>
this (notably the always-inliner and coroutine lowerer) are required for<br>
correctness, they need to run over everything.<br>
<br>
This just adds an extra layer of iteration over the CallGraph to keep<br>
track of which functions we've already visited and get the next batch of<br>
SCCs.<br>
<br>
Should fix PR38029.<br>
<br>
Modified:<br>
llvm/trunk/include/llvm/ADT/SC<wbr>CIterator.h<br>
llvm/trunk/lib/Analysis/CallGr<wbr>aphSCCPass.cpp<br>
llvm/trunk/test/Transforms/Inl<wbr>ine/always-inline.ll<br>
<br>
Modified: llvm/trunk/include/llvm/ADT/SC<wbr>CIterator.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ADT/SCCIterator.h?rev=336419&r1=336418&r2=336419&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/llvm/trunk/include/llvm/<wbr>ADT/SCCIterator.h?rev=336419&r<wbr>1=336418&r2=336419&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/include/llvm/ADT/SC<wbr>CIterator.h (original)<br>
+++ llvm/trunk/include/llvm/ADT/SC<wbr>CIterator.h Fri Jul 6 01:04:47 2018<br>
@@ -90,6 +90,7 @@ class scc_iterator : public iterator_fac<br>
/// Compute the next SCC using the DFS traversal.<br>
void GetNextSCC();<br>
<br>
+public:<br>
scc_iterator(NodeRef entryN) : visitNum(0) {<br>
DFSVisitOne(entryN);<br>
GetNextSCC();<br>
@@ -98,12 +99,6 @@ class scc_iterator : public iterator_fac<br>
/// End is when the DFS stack is empty.<br>
scc_iterator() = default;<br>
<br>
-public:<br>
- static scc_iterator begin(const GraphT &G) {<br>
- return scc_iterator(GT::getEntryNode(<wbr>G));<br>
- }<br>
- static scc_iterator end(const GraphT &) { return scc_iterator(); }<br>
-<br>
/// Direct loop termination test which is more efficient than<br>
/// comparison with \c end().<br>
bool isAtEnd() const {<br>
@@ -222,16 +217,25 @@ bool scc_iterator<GraphT, GT>::hasLoop()<br>
return false;<br>
}<br>
<br>
-/// Construct the begin iterator for a deduced graph type T.<br>
+/// Construct the begin iterator for a deduced graph type T, starting from its<br>
+/// entry node.<br>
template <class T> scc_iterator<T> scc_begin(const T &G) {<br>
- return scc_iterator<T>::begin(G);<br>
+ return scc_iterator<T>(GraphTraits<T><wbr>::getEntryNode(G));<br>
}<br>
<br>
-/// Construct the end iterator for a deduced graph type T.<br>
+/// Construct the begin iterator for a graph type T, starting from the specified<br>
+/// node.<br>
+template <class T, class U = GraphTraits<T>><br>
+scc_iterator<T, U> scc_begin(typename U::NodeRef N) {<br>
+ return scc_iterator<T>(N);<br>
+}<br>
+<br>
+ /// Construct the end iterator for a deduced graph type T.<br>
template <class T> scc_iterator<T> scc_end(const T &G) {<br>
- return scc_iterator<T>::end(G);<br>
+ return scc_iterator<T>();<br>
}<br>
<br>
+<br>
} // end namespace llvm<br>
<br>
#endif // LLVM_ADT_SCCITERATOR_H<br>
<br>
Modified: llvm/trunk/lib/Analysis/CallGr<wbr>aphSCCPass.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Analysis/CallGraphSCCPass.cpp?rev=336419&r1=336418&r2=336419&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/llvm/trunk/lib/Analysis/<wbr>CallGraphSCCPass.cpp?rev=33641<wbr>9&r1=336418&r2=336419&view=<wbr>diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/lib/Analysis/CallGr<wbr>aphSCCPass.cpp (original)<br>
+++ llvm/trunk/lib/Analysis/CallGr<wbr>aphSCCPass.cpp Fri Jul 6 01:04:47 2018<br>
@@ -17,6 +17,7 @@<br>
<br>
#include "llvm/Analysis/CallGraphSCCPas<wbr>s.h"<br>
#include "llvm/ADT/DenseMap.h"<br>
+#include "llvm/ADT/DenseSet.h"<br>
#include "llvm/ADT/SCCIterator.h"<br>
#include "llvm/ADT/Statistic.h"<br>
#include "llvm/Analysis/CallGraph.h"<br>
@@ -63,6 +64,10 @@ public:<br>
/// whether any of the passes modifies the module, and if so, return true.<br>
bool runOnModule(Module &M) override;<br>
<br>
+ /// Execute all of the passes scheduled for execution on a given SCC. Return<br>
+ /// true if any passes modify the module.<br>
+ bool runOnSCC(CallGraphSCC &SCC, CallGraph &CG);<br>
+<br>
using ModulePass::doInitialization;<br>
using ModulePass::doFinalization;<br>
<br>
@@ -448,51 +453,78 @@ bool CGPassManager::RunAllPassesOnS<wbr>CC(Ca<br>
bool CGPassManager::runOnModule(Mod<wbr>ule &M) {<br>
CallGraph &CG = getAnalysis<CallGraphWrapperPa<wbr>ss>().getCallGraph();<br>
bool Changed = doInitialization(CG);<br>
- <br>
- // Walk the callgraph in bottom-up SCC order.<br>
- scc_iterator<CallGraph*> CGI = scc_begin(&CG);<br>
<br>
- CallGraphSCC CurSCC(CG, &CGI);<br>
- while (!CGI.isAtEnd()) {<br>
- // Copy the current SCC and increment past it so that the pass can hack<br>
- // on the SCC if it wants to without invalidating our iterator.<br>
- const std::vector<CallGraphNode *> &NodeVec = *CGI;<br>
- CurSCC.initialize(NodeVec);<br>
- ++CGI;<br>
-<br>
- // At the top level, we run all the passes in this pass manager on the<br>
- // functions in this SCC. However, we support iterative compilation in the<br>
- // case where a function pass devirtualizes a call to a function. For<br>
- // example, it is very common for a function pass (often GVN or instcombine)<br>
- // to eliminate the addressing that feeds into a call. With that improved<br>
- // information, we would like the call to be an inline candidate, infer<br>
- // mod-ref information etc.<br>
- //<br>
- // Because of this, we allow iteration up to a specified iteration count.<br>
- // This only happens in the case of a devirtualized call, so we only burn<br>
- // compile time in the case that we're making progress. We also have a hard<br>
- // iteration count limit in case there is crazy code.<br>
- unsigned Iteration = 0;<br>
- bool DevirtualizedCall = false;<br>
- do {<br>
- LLVM_DEBUG(if (Iteration) dbgs()<br>
- << " SCCPASSMGR: Re-visiting SCC, iteration #" << Iteration<br>
- << '\n');<br>
- DevirtualizedCall = false;<br>
- Changed |= RunAllPassesOnSCC(CurSCC, CG, DevirtualizedCall);<br>
- } while (Iteration++ < MaxIterations && DevirtualizedCall);<br>
- <br>
- if (DevirtualizedCall)<br>
- LLVM_DEBUG(dbgs() << " CGSCCPASSMGR: Stopped iteration after "<br>
- << Iteration<br>
- << " times, due to -max-cg-scc-iterations\n");<br>
+ DenseSet<const Function *> Visited;<br>
<br>
- MaxSCCIterations.updateMax(Ite<wbr>ration);<br>
+ CallGraph::iterator INext;<br>
+ for (auto I = CG.begin(), E = CG.end(); I != E;) {<br>
+ // Walk the callgraph in bottom-up SCC order.<br>
+ auto CGI = scc_begin<CallGraph *>(I->second.get());<br>
+<br>
+ CallGraphSCC CurSCC(CG, &CGI);<br>
+ while (!CGI.isAtEnd()) {<br>
+ const std::vector<CallGraphNode *> &NodeVec = *CGI;<br>
+<br>
+ // Record that we've seen this set of functions so we don't run the pass<br>
+ // twice.<br>
+ for (auto *Elt : NodeVec)<br>
+ Visited.insert(Elt->getFunctio<wbr>n());<br>
+<br>
+ // Copy the current SCC and increment past it so that the pass can hack<br>
+ // on the SCC if it wants to without invalidating our iterator.<br>
+ CurSCC.initialize(NodeVec);<br>
+ ++CGI;<br>
+<br>
+ // If we've got all functions reachable from our chosen initial node,<br>
+ // calculate the next node we need to search from now before parts of the<br>
+ // graph are invalidated.<br>
+ if (CGI.isAtEnd()) {<br>
+ do<br>
+ ++I;<br>
+ while (I != E && Visited.count(I->first));<br>
+ }<br>
+<br>
+ Changed |= runOnSCC(CurSCC, CG);<br>
+ }<br>
}<br>
+<br>
Changed |= doFinalization(CG);<br>
return Changed;<br>
}<br>
<br>
+bool CGPassManager::runOnSCC(CallGr<wbr>aphSCC &CurSCC, CallGraph &CG) {<br>
+ bool Changed = false;<br>
+<br>
+ // At the top level, we run all the passes in this pass manager on the<br>
+ // functions in this SCC. However, we support iterative compilation in the<br>
+ // case where a function pass devirtualizes a call to a function. For<br>
+ // example, it is very common for a function pass (often GVN or instcombine)<br>
+ // to eliminate the addressing that feeds into a call. With that improved<br>
+ // information, we would like the call to be an inline candidate, infer<br>
+ // mod-ref information etc.<br>
+ //<br>
+ // Because of this, we allow iteration up to a specified iteration count.<br>
+ // This only happens in the case of a devirtualized call, so we only burn<br>
+ // compile time in the case that we're making progress. We also have a hard<br>
+ // iteration count limit in case there is crazy code.<br>
+ unsigned Iteration = 0;<br>
+ bool DevirtualizedCall = false;<br>
+ do {<br>
+ LLVM_DEBUG(if (Iteration) dbgs()<br>
+ << " SCCPASSMGR: Re-visiting SCC, iteration #" << Iteration<br>
+ << '\n');<br>
+ DevirtualizedCall = false;<br>
+ Changed |= RunAllPassesOnSCC(CurSCC, CG, DevirtualizedCall);<br>
+ } while (Iteration++ < MaxIterations && DevirtualizedCall);<br>
+<br>
+ if (DevirtualizedCall)<br>
+ LLVM_DEBUG(dbgs() << " CGSCCPASSMGR: Stopped iteration after " << Iteration<br>
+ << " times, due to -max-cg-scc-iterations\n");<br>
+<br>
+ MaxSCCIterations.updateMax(Ite<wbr>ration);<br>
+ return Changed;<br>
+}<br>
+<br>
/// Initialize CG<br>
bool CGPassManager::doInitializatio<wbr>n(CallGraph &CG) {<br>
bool Changed = false;<br>
<br>
Modified: llvm/trunk/test/Transforms/Inl<wbr>ine/always-inline.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/Inline/always-inline.ll?rev=336419&r1=336418&r2=336419&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-pr<wbr>oject/llvm/trunk/test/Transfor<wbr>ms/Inline/always-inline.ll?rev<wbr>=336419&r1=336418&r2=336419&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/test/Transforms/Inl<wbr>ine/always-inline.ll (original)<br>
+++ llvm/trunk/test/Transforms/Inl<wbr>ine/always-inline.ll Fri Jul 6 01:04:47 2018<br>
@@ -316,3 +316,10 @@ define void @outer14() {<br>
call void @inner14()<br>
ret void<br>
}<br>
+<br>
+define internal i32 @outer15() {<br>
+; CHECK-LABEL: @outer15<br>
+; CHECK: ret i32 1<br>
+ %res = call i32 @inner1()<br>
+ ret i32 %res<br>
+}<br>
<br>
<br>
______________________________<wbr>_________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>