<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Have you tried -time-passes to see where it’s actually spending time? I don’t think there’s currently a timer that covers printing global variables to assembly, but you should be able to rule out something else.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Currently, the fastest path for emitting global data into an object file is an i8 array “[1000000 x i8]”; given a module in memory, we make one extra copy over the ideal of just directly calling write() on the bits, which should be fast
enough for most purposes. A “[1000000 x float]” currently uses a less efficient path, which copies the values one by one, but it probably wouldn’t be hard to optimize. If you’re emitting something that isn’t just an array of constant data, it gets less efficient.
See emitGlobalConstantDataSequential in lib/CodeGen/AsmPrinter.cpp.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-Eli<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> llvm-dev <llvm-dev-bounces@lists.llvm.org> <b>On Behalf Of
</b>Chris Lovett via llvm-dev<br>
<b>Sent:</b> Wednesday, May 8, 2019 3:52 PM<br>
<b>To:</b> llvm-dev@lists.llvm.org<br>
<b>Subject:</b> [EXT] [llvm-dev] slow performance in llc.exe to do with large global floating point arrays<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">We are building a neural network compiler using LLVM, see
<a href="https://github.com/Microsoft/ELL">https://github.com/Microsoft/ELL</a>.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">We want to put the neural network weights into a bunch of global float arrays because it allows us to more easily leverage<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">Flash RAM on small embedded devices. For example, it enables these kinds of scenarios:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black"><a href="https://lovettchris.github.io/posts/keyword_spotting/">keyword spotting demo</a>.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">We are finding some pretty bad compiler performance in some cases. For example,
<a href="https://gist.github.com/lovettchris/91e30bce1d18f16eddaf67306101e4e0">this github gist</a> contains a bitcode file which is a neural network compiled by ELL and it has about 30mb of floating point data, and when we put that through llc it takes 262
seconds to compile (on an Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz), but if we strip out the weights the "code" component of our neural network inference takes only 2 seconds to compile.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">We’ve noticed a good improvement in LLVM 8.0 in this area, but we think there’s still a lot more that could be done. For example,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">is it possible to dump big arrays of global floating point data into a binary without invoking huge assembly writer overhead?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">Perhaps what is happening is the optimizer is trying to optimized away unused floats but we would like to disable that and just<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">tell the compiler dump the floats into the object file, don’t bother trying to optimize them….<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:Consolas;color:black">Any thoughts?<o:p></o:p></span></p>
</div>
</div>
</body>
</html>