<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Ah -- awesome news! Sounds like it may be fixed soon.</div><div dir="ltr"><br></div><div dir="ltr">Indeed, the dates in the early history of the svn repository did jump around a bit, because "clang" was imported from an external repository in 2007, while it had already been under development for a year.</div><div dir="ltr"><div><br></div><div>To be precise, the SVN revisions r38537 through r39730, except for r39142, were imported into the SVN repository, with their original commit dates. Those dates are thus out of order compared with surrounding commits. E.g. r38535 has the date 2007-07-11 08:47:55 +0000, while r38537 is from a year earlier, 2006-06-18 05:42:02 +0000.</div><div><br></div></div></div></div></div></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 2, 2019 at 11:16 AM Björn Pettersson A <<a href="mailto:bjorn.a.pettersson@ericsson.com">bjorn.a.pettersson@ericsson.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US">
<div class="gmail-m_2123302635732683738WordSection1">
<p class="MsoNormal">I asked about this on <a href="mailto:git@vger.kernel.org" target="_blank">git@vger.kernel.org</a>:<u></u><u></u></p>
<p class="MsoNormal"> <a href="https://public-inbox.org/git/20190402132756.GB13141@sigill.intra.peff.net/T/#m1fd5da534d39f967a8ce8b3361bc2e00b9214f31" target="_blank">
https://public-inbox.org/git/20190402132756.GB13141@sigill.intra.peff.net/T/#m1fd5da534d39f967a8ce8b3361bc2e00b9214f31</a><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I’ve already got an answer that we seem to be unlucky with some access patterns when doing “git log –parents” in the monorepo,<u></u><u></u></p>
<p class="MsoNormal">and that we hit some quadratic analysis of the commit history. Hopefully something they can fix (Jeff King already had some ideas).<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-top:none;border-right:none;border-bottom:none;border-left:1.5pt solid blue;padding:0cm 0cm 0cm 4pt">
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b>From:</b> James Y Knight <<a href="mailto:jyknight@google.com" target="_blank">jyknight@google.com</a>> <br>
<b>Sent:</b> den 27 mars 2019 20:38<br>
<b>To:</b> Björn Pettersson A <<a href="mailto:bjorn.a.pettersson@ericsson.com" target="_blank">bjorn.a.pettersson@ericsson.com</a>><br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] monorepo: bad performance when using gitk / git log<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">The problem here seems to be due to the combination of specifying --parents, and specifying a pathname to filter by. I can certainly reproduce a _remarkable_ slowness with that combination from git....<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">On my machine:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">$ time git log --parents --oneline origin/master > /dev/null</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">real 0m4.001s</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">$ time git log origin/master -- llvm/test/CodeGen/Generic/bswap.ll > /dev/null</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">real 0m5.332s</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black"><u></u> <u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">$ time git log --parents --oneline origin/master -- llvm/test/CodeGen/Generic/bswap.ll > /dev/null</span><u></u><u></u></p>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New";color:black">real 2m48.944s</span><u></u><u></u></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">That said, I use gitk frequently, and had not noticed performance issues. But, I'd never tried invoking it with a path on the command-line, only with ref names, so it's not hitting the bad case.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Nor have I noted issues with git log, but again, I'd never have run it with --parents, so I don't hit this bad case.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Maybe worth reporting as a possible bug to git? Surely whatever algorithm it's using shouldn't be _this_ slow.<u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Wed, Mar 27, 2019 at 9:23 AM Björn Pettersson A via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span lang="SV">Hi!</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="SV"> </span><u></u><u></u></p>
<p class="MsoNormal">Anyone else experiencing performance problems when using the new monorepo?<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">My experience is that performance of gitk (and git log) sometimes is really bad when working in the monorepo.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">I’ve mainly seen it when using gitk on specific files/directories, but since gitk seems to be using “git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD --
<file>” it is possible to observe the same thing when using git log.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">The problem can be seen when creating a brand new commit (with a new file):<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ git clone
<a href="https://github.com/llvm/llvm-project.git" target="_blank">https://github.com/llvm/llvm-project.git</a> llvm-project<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ cd llvm-project<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ touch dummy<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ git add dummy<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ git commit -m "test"<u></u><u></u></p>
<p class="MsoNormal">[master 6539b74dd0e] test<u></u><u></u></p>
<p class="MsoNormal">1 file changed, 0 insertions(+), 0 deletions(-)<u></u><u></u></p>
<p class="MsoNormal">create mode 100644 llvm/dummy<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- dummy > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">198.37user 0.40system 3:18.67elapsed 100%CPU (0avgtext+0avgdata 696456maxresident)k</span><u></u><u></u></p>
<p class="MsoNormal">0inputs+0outputs (0major+175765minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">But also when examining older files, here are some tests using the monorepo:<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ git clone
<a href="https://github.com/llvm/llvm-project.git" target="_blank">https://github.com/llvm/llvm-project.git</a> llvm-project<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ cd llvm-project<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">5.15user 0.26system 0:05.42elapsed 99%CPU (0avgtext+0avgdata 220344maxresident)k</span><u></u><u></u></p>
<p class="MsoNormal">0inputs+0outputs (0major+56131minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- README.md > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">155.20user 0.34system 2:35.45elapsed 100%CPU (0avgtext+0avgdata 636744maxresident)k</span><u></u><u></u></p>
<p class="MsoNormal">0inputs+0outputs (0major+160862minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- llvm/CODE_OWNERS.TXT > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">55.48user 0.34system 0:55.80elapsed 100%CPU (0avgtext+0avgdata 690124maxresident)k</span><u></u><u></u></p>
<p class="MsoNormal">0inputs+0outputs (0major+174196minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- llvm/test/CodeGen/Generic/bswap.ll > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">192.97user 0.33system 3:13.19elapsed 100%CPU (0avgtext+0avgdata 696496maxresident)k</span><u></u><u></u></p>
<p class="MsoNormal">0inputs+0outputs (0major+176003minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Same tests when using the old llvm repo (there is no README.md so I skipped that test here):<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">2.72user 0.12system 0:02.84elapsed 99%CPU (0avgtext+0avgdata 136628maxresident)k</span><u></u><u></u></p>
<p class="MsoNormal">0inputs+0outputs (0major+36354minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- CODE_OWNERS.TXT > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">2.74user 0.19system 0:02.93elapsed 99%CPU (0avgtext+0avgdata 344756maxresident)k</span><u></u><u></u></p>
<p class="MsoNormal">0inputs+0outputs (0major+88975minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- test/CodeGen/Generic/bswap.ll > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">3.76user 0.19system 0:03.96elapsed 99%CPU (0avgtext+0avgdata 380416maxresident)k</span><u></u><u></u></p>
<p class="MsoNormal">0inputs+0outputs (0major+98218minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">The example with test/CodeGen/Generic/bswap.ll indicates that it can take 193/4=48 times longer time to open gitk (or run git log) on a file when using the monorepo(!?!?).<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">I’m not so familiar with the inner details of git. Could this be a bad repack of the llvm-projects repo or something?<u></u><u></u></p>
<p class="MsoNormal">Or is it just that we now squeeze so many commits into the same repo that I should expect the performance to be even worse in the future?<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">The figures above is when using git 2.14.1, but I’ve also tried 2.20.0 with similar results.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Regards,<u></u><u></u></p>
<p class="MsoNormal">Björn<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><u></u><u></u></p>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote></div>