<div dir="ltr"><div dir="ltr"><div>The problem here seems to be due to the combination of specifying --parents, and specifying a pathname to filter by. I can certainly reproduce a _remarkable_ slowness with that combination from git....</div><div><br></div><div><div>On my machine:</div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0)">$ time git log --parents --oneline origin/master > /dev/null</span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0)">real 0m4.001s</span><br></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0)"><br></span></span></div><div><div><font color="#000000" face="monospace">$ time git log origin/master -- llvm/test/CodeGen/Generic/bswap.ll > /dev/null</font></div><div><font color="#000000" face="monospace">real 0m5.332s</font></div><div style="color:rgb(0,0,0);font-family:monospace"><br></div></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0)">$ time git log --parents --oneline origin/master -- llvm/test/CodeGen/Generic/bswap.ll > /dev/null</span><br></span><div><span style="font-family:monospace"><span style="color:rgb(0,0,0)">real 2m48.944s</span><br></span></div></div></div><div><br></div><div dir="ltr">That said, I use gitk frequently, and had not noticed performance issues. But, I'd never tried invoking it with a path on the command-line, only with ref names, so it's not hitting the bad case.</div><div dir="ltr"><br></div><div dir="ltr">Nor have I noted issues with git log, but again, I'd never have run it with --parents, so I don't hit this bad case.</div><div dir="ltr"><br></div><div>Maybe worth reporting as a possible bug to git? Surely whatever algorithm it's using shouldn't be _this_ slow.</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 27, 2019 at 9:23 AM Björn Pettersson A via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US">
<div class="gmail-m_6718864991168543156WordSection1">
<p class="MsoNormal"><span lang="SV">Hi!<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="SV"><u></u> <u></u></span></p>
<p class="MsoNormal">Anyone else experiencing performance problems when using the new monorepo?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">My experience is that performance of gitk (and git log) sometimes is really bad when working in the monorepo.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I’ve mainly seen it when using gitk on specific files/directories, but since gitk seems to be using “git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- <file>” it is possible to observe the same thing when using
git log.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The problem can be seen when creating a brand new commit (with a new file):<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ git clone <a href="https://github.com/llvm/llvm-project.git" target="_blank">https://github.com/llvm/llvm-project.git</a> llvm-project<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ cd llvm-project<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ touch dummy<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ git add dummy<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ git commit -m "test"<u></u><u></u></p>
<p class="MsoNormal">[master 6539b74dd0e] test<u></u><u></u></p>
<p class="MsoNormal">1 file changed, 0 insertions(+), 0 deletions(-)<u></u><u></u></p>
<p class="MsoNormal">create mode 100644 llvm/dummy<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- dummy > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">198.37user 0.40system 3:18.67elapsed 100%CPU (0avgtext+0avgdata 696456maxresident)k<u></u><u></u></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+175765minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">But also when examining older files, here are some tests using the monorepo:<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ git clone <a href="https://github.com/llvm/llvm-project.git" target="_blank">https://github.com/llvm/llvm-project.git</a> llvm-project<u></u><u></u></p>
<p class="MsoNormal">bash-4.1$ cd llvm-project<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">5.15user 0.26system 0:05.42elapsed 99%CPU (0avgtext+0avgdata 220344maxresident)k<u></u><u></u></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+56131minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- README.md > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">155.20user 0.34system 2:35.45elapsed 100%CPU (0avgtext+0avgdata 636744maxresident)k<u></u><u></u></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+160862minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- llvm/CODE_OWNERS.TXT > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">55.48user 0.34system 0:55.80elapsed 100%CPU (0avgtext+0avgdata 690124maxresident)k<u></u><u></u></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+174196minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- llvm/test/CodeGen/Generic/bswap.ll > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">192.97user 0.33system 3:13.19elapsed 100%CPU (0avgtext+0avgdata 696496maxresident)k<u></u><u></u></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+176003minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Same tests when using the old llvm repo (there is no README.md so I skipped that test here):<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">2.72user 0.12system 0:02.84elapsed 99%CPU (0avgtext+0avgdata 136628maxresident)k<u></u><u></u></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+36354minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- CODE_OWNERS.TXT > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">2.74user 0.19system 0:02.93elapsed 99%CPU (0avgtext+0avgdata 344756maxresident)k<u></u><u></u></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+88975minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- test/CodeGen/Generic/bswap.ll > /dev/null<u></u><u></u></p>
<p class="MsoNormal"><span lang="SV">3.76user 0.19system 0:03.96elapsed 99%CPU (0avgtext+0avgdata 380416maxresident)k<u></u><u></u></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+98218minor)pagefaults 0swaps<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The example with test/CodeGen/Generic/bswap.ll indicates that it can take 193/4=48 times longer time to open gitk (or run git log) on a file when using the monorepo(!?!?).<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I’m not so familiar with the inner details of git. Could this be a bad repack of the llvm-projects repo or something?<u></u><u></u></p>
<p class="MsoNormal">Or is it just that we now squeeze so many commits into the same repo that I should expect the performance to be even worse in the future?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The figures above is when using git 2.14.1, but I’ve also tried 2.20.0 with similar results.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Regards,<u></u><u></u></p>
<p class="MsoNormal">Björn<u></u><u></u></p>
</div>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>