<div dir="ltr"><div class="gmail-phui-timeline-core-content"><span class="gmail-transaction-comment"><div class="gmail-phabricator-remarkup"><h2 class="gmail-remarkup-header">Machine IR Profile (MIP)</h2><div><a href="https://reviews.llvm.org/D104060">https://reviews.llvm.org/D104060</a></div><div>The full branch can be found at <a href="https://github.com/ellishg/llvm-project" class="gmail-remarkup-link" target="_blank" rel="noreferrer">https://github.com/ellishg/llvm-project</a></div>
<h4 class="gmail-remarkup-header">tl;dr;</h4>
<p>This is a proposal to introduce a new instrumentation pass that can
produce optimization profiles with a focus on binary size and runtime
performance of the instrumented binaries.</p>
<p>Our instrumented binaries record machine function call counts,
machine function timestamps, machine basic block coverage, and a subset
of the dynamic call graph. There is also a more lightweight mode that
only collects machine function coverage data that has negligible runtime
overhead and a binary size increase of 2-5% for instrumented binaries.</p><h3 class="gmail-remarkup-header">Motivation</h3>
<p>In the mobile space, increasing binary size has an outsized impact on
both runtime performance and download speed. Current instrumentation
implementations such as XRay and GCov produce binaries that are too slow
and too large to run on real mobile devices. We propose a new pass that
injects instrumentation code at the machine ir level. At runtime, we
write profile data to our custom <tt class="gmail-remarkup-monospaced">__llvm_mipraw</tt> section that is eventually dumped to a <tt class="gmail-remarkup-monospaced">.mipraw</tt> file. At buildtime, we emit a <tt class="gmail-remarkup-monospaced">.mipmap</tt> file which we use to map function information to data in the <tt class="gmail-remarkup-monospaced">.mipraw</tt>
file. The result is that no redundant function info is stored in the
binary, which allows our instrumentation to have minimal size overhead.</p>
<p>MIP has been implemented on ELF and Mach-O targets for x86_64, AArch64, and Armv7 with Thumb and Thumb2.</p>
<h3 class="gmail-remarkup-header">Performance</h3>
<p>Our focus for now is on the performance and size of binaries that
have injected instrumentation instead of binaries that have been
optimized with instrumentation profiles. We collected some basic results
from <tt class="gmail-remarkup-monospaced">MultiSource/Benchmarks</tt> in <tt class="gmail-remarkup-monospaced">llvm-test-suite</tt> for both MIP and clang’s instrumentation using the <tt class="gmail-remarkup-monospaced">-fprofile-generate</tt>
flag. It should be noted that this comparison is not fair because
clang’s instrumentation collects much more data than just function
coverage. However, we expect fully-featured MIP to have similar metrics.</p>
<h4 class="gmail-remarkup-header">Instrumented Binary Size</h4>
<p>At the moment, we have implemented function coverage which injects
one x86_64 instruction (7 bytes) and one byte of global data for each
instrumented function, which should have minimal impact on binary size
and runtime performance. In fact, our results show that we should expect
MIP instrumented binaries to be only 2-5% larger. We contrast this with
clang’s instrumentation, which can increase the binary size by
500-900%.</p>
<h4 class="gmail-remarkup-header">Instrumented Execution Time</h4>
<p>We found that MIP had negligable execution time regressions when
instrumented with MIP. Again, we can (unfairly) contrast this to <tt class="gmail-remarkup-monospaced">-fprofile-generate</tt> which increased execution time by 1-40%.</p>
<h3 class="gmail-remarkup-header">Usage</h3>
<p>We use the <tt class="gmail-remarkup-monospaced">-fmachine-profile-generate</tt> clang flag to produce an instrumented binary and then use <tt class="gmail-remarkup-monospaced">llvm-objcopy</tt> to extract the <tt class="gmail-remarkup-monospaced">.mipmap</tt> file.</p>
<div class="gmail-remarkup-code-block"><pre class="gmail-remarkup-code"><span class="gmail-gp">$ clang -g -fmachine-profile-generate main.cpp</span>
<span class="gmail-gp">$ llvm-objcopy --dump-section=__llvm_mipmap=default.mipmap a.out /dev/null</span>
<span class="gmail-gp">$ llvm-strip -g a.out -o a.out.stripped</span></pre></div>
<p>This will produce the instrumented binary a.out and a map file <tt class="gmail-remarkup-monospaced">default.mipmap</tt>.</p>
<p>When we run the binary, it will produce a <tt class="gmail-remarkup-monospaced">default.mipraw</tt> file containing the profile data for that run.</p>
<div class="gmail-remarkup-code-block"><pre class="gmail-remarkup-code"><span class="gmail-gp">$ ./a.out.stripped</span>
<span class="gmail-gp">$ ls</span>
<span class="gmail-go">a.out default.mipmap default.mipraw main.cpp</span></pre></div>
<p>Then we use our custom tool to postprocess the raw profile and produce the final profile <tt class="gmail-remarkup-monospaced">default.mip</tt>.</p>
<div class="gmail-remarkup-code-block"><pre class="gmail-remarkup-code"><span class="gmail-gp">$ llvm-mipdata create -p default.mip default.mipmap</span>
<span class="gmail-gp">$ llvm-mipdata merge -p default.mip default.mipraw</span></pre></div>
<p>If our binary has debug info, we can use it to report source information along with the profile data.</p>
<div class="gmail-remarkup-code-block"><pre class="gmail-remarkup-code"><span class="gmail-gp">$ llvm-mipdata show -p default.mip --debug a.out</span>
<span class="gmail-go">_Z3fooi</span>
<span class="gmail-go"> Source Info: /home/main.cpp:9</span>
<span class="gmail-go"> Call Count: 0</span>
<span class="gmail-go"> Block Coverage:</span>
<span class="gmail-go"> COLD COLD COLD COLD COLD</span>
<span class="gmail-go"></span>
<span class="gmail-go">_Z3bari</span>
<span class="gmail-go"> Source Info: /home/main.cpp:16</span>
<span class="gmail-go"> Call Count: 1</span>
<span class="gmail-go"> Block Coverage:</span>
<span class="gmail-go"> HOT HOT COLD HOT HOT</span></pre></div>
<p>Finally, we can consume the profile using the clang flag <tt class="gmail-remarkup-monospaced">-fmachine-profile-use=</tt> to produce a profile-optimized binary.</p>
<div class="gmail-remarkup-code-block"><pre class="gmail-remarkup-code"><span class="gmail-gp">$ clang -fmachine-profile-use=default.mip main.cpp</span></pre></div></div></span></div></div>