<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi,<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 9 Feb 2020, at 03:28, Mehdi AMINI via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class="">Hi,<div class=""><br class=""></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 8, 2020 at 5:17 AM Nikita Popov via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><div class="">Hi,</div><div class=""><br class=""></div><div class="">Does the LLVM project perform any kind of tracking for commit-by-commit compile-time changes? It looks like LNT only tracks run-time performance (and to be honest I wasn't able to make heads or tails of the results even for that -- the interface was pretty unintuitive to me.)<br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">At some point Apple was feeding compile-time data to LNT, but it seems like the jobs aren't active anymore: <a href="http://green.lab.llvm.org/green/view/Compile Time/" class="">http://green.lab.llvm.org/green/view/Compile%20Time/</a></div></div></div></div></div></div></blockquote><br class=""></div><div>Yes, it looks like the matrix view is broken. One of the relevant jobs is <a href="http://104.154.54.203/db_default/v4/nts/machine/1353" class="">http://104.154.54.203/db_default/v4/nts/machine/1353</a> which has a fairly recent submission <a href="http://104.154.54.203/db_default/v4/nts/131539" class="">http://104.154.54.203/db_default/v4/nts/131539</a> </div><div><br class=""></div><div>But it looks like we need to adjust the ‘Run order’. It is currently using git commit hashes, which messes up the ordering of submissions. And submissions by the bot seem to be very irregular, due to various failures.</div><div><br class=""></div><div><br class=""></div><br class=""><div class=""><div class=""></div><blockquote type="cite" class=""><div class="">While it is "normal" that each new LLVM release regresses compile-time by 2-3%, LLVM 10 seems to be going for a new record here, with optimized builds regressing by around 5-15%, and that's already after mitigating the largest regression: <a href="https://perf.rust-lang.org/compare.html?start=58b834344fc7b9185e7a50db1ff24e5eb07dae5e&end=f9c17b2d9bd4ca61bb0eb8b226dd61aaf254156c" target="_blank" class="">https://perf.rust-lang.org/compare.html?start=58b834344fc7b9185e7a50db1ff24e5eb07dae5e&end=f9c17b2d9bd4ca61bb0eb8b226dd61aaf254156c</a></div><div class=""><br class=""></div><div class="">It does not seem like these regressions are specific to Rust either, see for example <a href="https://bugs.llvm.org/show_bug.cgi?id=44408" target="_blank" class="">https://bugs.llvm.org/show_bug.cgi?id=44408</a>.</div><div class=""><br class=""></div><div class="">The Rust project tracks compile-time results for each commit (see link above), and thanks to that can quickly spot and revert changes that negatively affect performance. I think it would be really great if something similar would exist for LLVM...<br class=""></div><div class=""><br class=""></div></blockquote><br class=""></div><div class="">The Rust tracking looks awesome! It’s great that we get feedback from there, thanks for keeping an eye on that.</div><div class=""><br class=""></div><div class="">In theory, the bots on green dragon should submit CTMark numbers regularly for CTMark on a few configurations (although not for every commit), but as mentioned earlier there are a few issues that crept in. I’ll try to reach out to the right people to have a look.</div><div class=""><br class=""></div><div class="">Cheers,</div><div class="">Florian</div></body></html>