<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span lang="SV">Hi!<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="SV"><o:p> </o:p></span></p>
<p class="MsoNormal">Anyone else experiencing performance problems when using the new monorepo?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My experience is that performance of gitk (and git log) sometimes is really bad when working in the monorepo.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’ve mainly seen it when using gitk on specific files/directories, but since gitk seems to be using “git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- <file>” it is possible to observe the same thing when using
git log.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The problem can be seen when creating a brand new commit (with a new file):<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ git clone https://github.com/llvm/llvm-project.git llvm-project<o:p></o:p></p>
<p class="MsoNormal">bash-4.1$ cd llvm-project<o:p></o:p></p>
<p class="MsoNormal">bash-4.1$ touch dummy<o:p></o:p></p>
<p class="MsoNormal">bash-4.1$ git add dummy<o:p></o:p></p>
<p class="MsoNormal">bash-4.1$ git commit -m "test"<o:p></o:p></p>
<p class="MsoNormal">[master 6539b74dd0e] test<o:p></o:p></p>
<p class="MsoNormal">1 file changed, 0 insertions(+), 0 deletions(-)<o:p></o:p></p>
<p class="MsoNormal">create mode 100644 llvm/dummy<o:p></o:p></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- dummy > /dev/null<o:p></o:p></p>
<p class="MsoNormal"><span lang="SV">198.37user 0.40system 3:18.67elapsed 100%CPU (0avgtext+0avgdata 696456maxresident)k<o:p></o:p></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+175765minor)pagefaults 0swaps<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">But also when examining older files, here are some tests using the monorepo:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ git clone https://github.com/llvm/llvm-project.git llvm-project<o:p></o:p></p>
<p class="MsoNormal">bash-4.1$ cd llvm-project<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD > /dev/null<o:p></o:p></p>
<p class="MsoNormal"><span lang="SV">5.15user 0.26system 0:05.42elapsed 99%CPU (0avgtext+0avgdata 220344maxresident)k<o:p></o:p></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+56131minor)pagefaults 0swaps<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- README.md > /dev/null<o:p></o:p></p>
<p class="MsoNormal"><span lang="SV">155.20user 0.34system 2:35.45elapsed 100%CPU (0avgtext+0avgdata 636744maxresident)k<o:p></o:p></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+160862minor)pagefaults 0swaps<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- llvm/CODE_OWNERS.TXT > /dev/null<o:p></o:p></p>
<p class="MsoNormal"><span lang="SV">55.48user 0.34system 0:55.80elapsed 100%CPU (0avgtext+0avgdata 690124maxresident)k<o:p></o:p></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+174196minor)pagefaults 0swaps<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- llvm/test/CodeGen/Generic/bswap.ll > /dev/null<o:p></o:p></p>
<p class="MsoNormal"><span lang="SV">192.97user 0.33system 3:13.19elapsed 100%CPU (0avgtext+0avgdata 696496maxresident)k<o:p></o:p></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+176003minor)pagefaults 0swaps<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Same tests when using the old llvm repo (there is no README.md so I skipped that test here):<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD > /dev/null<o:p></o:p></p>
<p class="MsoNormal"><span lang="SV">2.72user 0.12system 0:02.84elapsed 99%CPU (0avgtext+0avgdata 136628maxresident)k<o:p></o:p></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+36354minor)pagefaults 0swaps<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- CODE_OWNERS.TXT > /dev/null<o:p></o:p></p>
<p class="MsoNormal"><span lang="SV">2.74user 0.19system 0:02.93elapsed 99%CPU (0avgtext+0avgdata 344756maxresident)k<o:p></o:p></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+88975minor)pagefaults 0swaps<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">bash-4.1$ /usr/bin/time git log --no-color -z --pretty=raw --show-notes --parents --boundary HEAD -- test/CodeGen/Generic/bswap.ll > /dev/null<o:p></o:p></p>
<p class="MsoNormal"><span lang="SV">3.76user 0.19system 0:03.96elapsed 99%CPU (0avgtext+0avgdata 380416maxresident)k<o:p></o:p></span></p>
<p class="MsoNormal">0inputs+0outputs (0major+98218minor)pagefaults 0swaps<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The example with test/CodeGen/Generic/bswap.ll indicates that it can take 193/4=48 times longer time to open gitk (or run git log) on a file when using the monorepo(!?!?).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m not so familiar with the inner details of git. Could this be a bad repack of the llvm-projects repo or something?<o:p></o:p></p>
<p class="MsoNormal">Or is it just that we now squeeze so many commits into the same repo that I should expect the performance to be even worse in the future?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The figures above is when using git 2.14.1, but I’ve also tried 2.20.0 with similar results.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards,<o:p></o:p></p>
<p class="MsoNormal">Björn<o:p></o:p></p>
</div>
</body>
</html>