<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
mso-fareast-language:EN-US;}
span.EmailStyle25
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:590889224;
mso-list-template-ids:54141268;}
@list l1
{mso-list-id:1288318626;
mso-list-template-ids:-1849384038;}
@list l1:level1
{mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level2
{mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level3
{mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level4
{mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level5
{mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level6
{mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level7
{mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level8
{mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level9
{mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l2
{mso-list-id:1524057697;
mso-list-type:hybrid;
mso-list-template-ids:-2052813660 134807553 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">A few points:<o:p></o:p></p>
<ul style="margin-top:0cm" type="disc">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l2 level1 lfo4">This doesn't need to be an official LLVM distribution, it can just be a thing that is exists in good faith that you can download if you want but comes with no guarantees of any kind.
A colleague who I was speaking to partially likened it to how people use Homebrew and install packages compiled by “someone”. Package managers do implement additional hash checks, etc. but are not very different from what I am suggesting.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l2 level1 lfo4">We can leave it up to the user to make sure that they have the same version of Clang, Cmake, etc. as the buildbot that compiled the code (the buildbots can store this information in
a text file and give it to the users). Like I said this would be a thing that exists with no guarantees of any kind. My idea is to just give people the resources that they need in order to contribute to LLVM and cannot get without spending money.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l2 level1 lfo4">While I do have access to SN-DBS, I’m not talking about myself personally here. People cannot be expected to have access to distributed build systems. Most university students for
example will have thin and light ultrabooks that are easy to carry to lectures and most definitely won’t be able to compile LLVM from scratch.<o:p></o:p></li></ul>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal">Nabeel<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="mso-fareast-language:EN-GB">From:</span></b><span lang="EN-US" style="mso-fareast-language:EN-GB"> Philip Reames <listmail@philipreames.com>
<br>
<b>Sent:</b> 14 April 2021 23:39<br>
<b>To:</b> Alexandre Ganea <alexandre.ganea@ubisoft.com>; Omer, Nabeel <Nabeel.Omer@sony.com><br>
<b>Cc:</b> llvm-dev@lists.llvm.org<br>
<b>Subject:</b> Re: [llvm-dev] Consider making build directories of the buildbots available<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p>One bit of warning, build trees tend to have relative paths baked in. From experience, trying to copy an llvm build tree from one location to another tends to not work well.<o:p></o:p></p>
<p>The best workaround I've seen for this type of thing is to use docker (or your container format of choice) to construct a "build image" with the full source and build trees, then update the source and trigger an (incremental) build.<o:p></o:p></p>
<p>After that's out of the way, you run into two major problems quickly.<o:p></o:p></p>
<p>1) Such build images tend to be gigantic. (50+gb) Bandwidth costs, storage costs, and download times add up *quickly*.
<br>
2) Their value as incremental build sources tend to age very quickly. If you look at the commits to llvm, core headers get changed shocking often. As such, if you're trying to follow ToT you quickly end up doing what is effectively a clean build anyways.
The only workflow which "somewhat works" is to develop against some recent snapshot, then rebase only at the very last and pay the cost of a full build.<o:p></o:p></p>
<p>I've played enough with ideas like this in the past to recommend you not go down this path.<o:p></o:p></p>
<p>An alternate approach I recommend - and use personally - is to do all builds on an AWS instance. With some basic scripting (<a href="https://github.com/preames/llvm-aws-builder">here's mine</a>), you can do fast builds for a couple of dollars a day. I've
been working this was for about 6 months, and have found it dramatically easier than all the options I played with before.
<o:p></o:p></p>
<p>Philip<o:p></o:p></p>
<div>
<p class="MsoNormal">On 4/14/21 1:53 PM, Alexandre Ganea via llvm-dev wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">That’s an interesting idea. There are several issues to consider:<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<ol style="margin-top:0cm" start="1" type="1">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l1 level1 lfo3">The build needs to be deterministic, if we want to share .OBJ files among LLVM developers. In essence, that means fully implementing [1]. I’m not sure how much of the LLVM codebase
supports that.<br>
Just the toolchain part is challenging to implement. We would need to ensure all users are using precisely the same compiler, the same linker, same cmake, same platform SDKs, etc. in order to expect good cache hits.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l1 level1 lfo3">There’s the security consideration. If anyone is to pull on the cached .OBJ files, you need to “trust” these .OBJ files in the first place. That means maybe restraining the list of
cache “publishers” to the LLVM github group, and signing the .OBJs with a private key or something along those lines.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l1 level1 lfo3">There’s a file size consideration. My build folder for only { llvm, clang, lld } is about 40 GB. When using the ThinLTO cache, that goes over 100 GB. Still, build artefacts compress
quite well (3:1 at least), and you’d probably pay the network price only once, then cache hits would be incremental.<o:p></o:p></li></ol>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">I think the same idea could apply for distributing the compilation. It’d be interesting to have a public LLVM distributed compilation service. But how can we trust it? Even if we only compile on “trusted” machines, there’s still a risk
of attack. That’s probably why these caching/remote compilation systems are only used inside an organization, which can guarantee trust (somehow). Since you’re at Sony, perhaps you have access to a internal SN-DBS pool? [2]<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">[1] <a href="https://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html">
https://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html</a><o:p></o:p></p>
<p class="MsoNormal">[2] <a href="https://www.snsystems.com/tech-blog/2014/01/06/building-with-the-network/">
<span lang="EN-CA">https://www.snsystems.com/tech-blog/2014/01/06/building-with-the-network/</span></a><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="FR" style="mso-fareast-language:EN-CA">De :</span></b><span lang="FR" style="mso-fareast-language:EN-CA"> llvm-dev
</span><a href="mailto:llvm-dev-bounces@lists.llvm.org"><span lang="FR" style="mso-fareast-language:EN-CA"><llvm-dev-bounces@lists.llvm.org></span></a><span style="mso-fareast-language:EN-CA">
<b><span lang="FR">De la part de</span></b><span lang="FR"> via llvm-dev<br>
<b>Envoyé :</b> April 14, 2021 3:31 PM<br>
<b>À :</b> </span></span><a href="mailto:llvm-dev@lists.llvm.org"><span lang="FR" style="mso-fareast-language:EN-CA">llvm-dev@lists.llvm.org</span></a><span lang="FR" style="mso-fareast-language:EN-CA"><br>
<b>Objet :</b> Re: [llvm-dev] Consider making build directories of the buildbots available</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Ah, apologies, I think I wasn’t clear. I am talking about making object files available in this manner so that people can download them and compile just their changes without having to compile all of LLVM, thus reducing the barrier to
entry. As far as I am aware, the releases on Github do not contain object files.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Nabeel Omer<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="mso-fareast-language:EN-GB">From:</span></b><span lang="EN-US" style="mso-fareast-language:EN-GB"> llvm-dev <</span><a href="mailto:llvm-dev-bounces@lists.llvm.org"><span lang="EN-US" style="mso-fareast-language:EN-GB">llvm-dev-bounces@lists.llvm.org</span></a><span lang="EN-US" style="mso-fareast-language:EN-GB">>
<b>On Behalf Of </b>Neil Nelson via llvm-dev<br>
<b>Sent:</b> 14 April 2021 20:05<br>
<b>To:</b> </span><a href="mailto:llvm-dev@lists.llvm.org"><span lang="EN-US" style="mso-fareast-language:EN-GB">llvm-dev@lists.llvm.org</span></a><span lang="EN-US" style="mso-fareast-language:EN-GB"><br>
<b>Subject:</b> Re: [llvm-dev] Consider making build directories of the buildbots available</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p><span style="font-size:10.0pt">Perhaps these pages may help.</span><o:p></o:p></p>
<p><a href="https://github.com/llvm/llvm-project/releases"><span style="font-size:10.0pt">https://github.com/llvm/llvm-project/releases</span></a><o:p></o:p></p>
<p><a href="https://github.com/llvm/llvm-project/releases/tag/llvmorg-12.0.0-rc5"><span style="font-size:10.0pt">https://github.com/llvm/llvm-project/releases/tag/llvmorg-12.0.0-rc5</span></a><o:p></o:p></p>
<p><span style="font-size:10.0pt">Neil Nelson</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt">On 4/14/21 12:53 PM, via llvm-dev wrote:</span><o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">Hi LLVM devs,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">As you are already aware, performing a clean build of LLVM requires considerable computing resources. This presents a barrier to entry for people who do not have access to large computers. Since the
<span lang="EN-US">buildbots already regularly compile the LLVM codebase, making tarballs of their build directories available on a public facing server will dramatically reduce the barrier to entry. Is this something that the community is willing to consider?</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Thanks,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Nabeel Omer</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="mso-fareast-language:EN-GB"> </span><o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>LLVM Developers mailing list<o:p></o:p></pre>
<pre><a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a><o:p></o:p></pre>
<pre><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><br>
<br>
<o:p></o:p></span></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>LLVM Developers mailing list<o:p></o:p></pre>
<pre><a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a><o:p></o:p></pre>
<pre><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p></o:p></pre>
</blockquote>
</div>
</body>
</html>