<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 28, 2019 at 11:03 AM Aditya K via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
> The splitting pass currently doesn’t move cold symbols into a separate section. Is that affecting your results? </div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Maybe partly, the main reason is that, in the absence of good profile info, we aren't finding many cold blocks.</div></div></blockquote><div><br></div><div>We noticed that the split cold functions are ending up in the regular .text section instead of .text.unlikely. Since that is done much later than splitting and is based on profile counts, it must be the case that profile data is not being propagated to the split functions in some way - do you know offhand if they are getting function_entry_count prof metadata?</div><div><br></div><div>The other thing we noticed is that the .text.unlikely section is also reducing significantly, so it seems like some of the already cold blocks are getting split - has anyone noticed that?</div><div><br></div><div>Teresa</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div>
<div id="gmail-m_1036890746296332442appendonsend"></div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
-Aditya</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_1036890746296332442divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> <a href="mailto:vsk@apple.com" target="_blank">vsk@apple.com</a> <<a href="mailto:vsk@apple.com" target="_blank">vsk@apple.com</a>> on behalf of Vedant Kumar <<a href="mailto:vedant_kumar@apple.com" target="_blank">vedant_kumar@apple.com</a>><br>
<b>Sent:</b> Monday, January 28, 2019 1:00 PM<br>
<b>To:</b> Aditya K<br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>; Sebastian Pop<br>
<b>Subject:</b> Re: [llvm-dev] Status update on the hot/cold splitting pass</font>
<div> </div>
</div>
<div style="overflow-wrap: break-word;">The splitting pass currently doesn’t move cold symbols into a separate section. Is that affecting your results?
</div>
<div style="overflow-wrap: break-word;">
<div><br>
</div>
<div>On Darwin, we plan on using a symbol attribute to provide an ordering hint to the linker (see r352227, N_COLD_FUNC).</div>
<div><br>
</div>
<div>vedant</div>
<div>
<div><br>
<blockquote type="cite">
<div>On Jan 28, 2019, at 10:51 AM, Aditya K via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:</div>
<br class="gmail-m_1036890746296332442x_Apple-interchange-newline">
<div>
<div style="font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;font-family:Calibri,Helvetica,sans-serif;font-size:12pt">
Very happy to see good results. On our side, we are still struggling with getting a good profile to get aggressive hot-cold splitting. Static profile isn't helping much for our use cases. I'll be curious to know if someone got good improvements only with static
profile analysis.<br>
</div>
<div style="font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;font-family:Calibri,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;font-family:Calibri,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;font-family:Calibri,Helvetica,sans-serif;font-size:12pt">
-Aditya<br>
</div>
<div style="color:rgb(255,255,255);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<div id="gmail-m_1036890746296332442x_appendonsend"></div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt"><br>
</div>
<hr style="display:inline-block;width:1077.02px">
<div id="gmail-m_1036890746296332442x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif"><b>From:</b><span class="gmail-m_1036890746296332442x_Apple-converted-space"> </span><a href="mailto:vsk@apple.com" target="_blank">vsk@apple.com</a> <<a href="mailto:vsk@apple.com" target="_blank">vsk@apple.com</a>>
on behalf of Vedant Kumar <<a href="mailto:vedant_kumar@apple.com" target="_blank">vedant_kumar@apple.com</a>><br>
<b>Sent:</b><span class="gmail-m_1036890746296332442x_Apple-converted-space"> </span>Friday, January 25, 2019 6:29 PM<br>
<b>To:</b><span class="gmail-m_1036890746296332442x_Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<b>Cc:</b><span class="gmail-m_1036890746296332442x_Apple-converted-space"> </span>Aditya Kumar; Sebastian Pop; Teresa Johnson; <a href="mailto:jun.l@samsung.com" target="_blank">jun.l@samsung.com</a>; Duncan Smith; Gerolf Hoflehner<br>
<b>Subject:</b><span class="gmail-m_1036890746296332442x_Apple-converted-space"> </span>Status update on the hot/cold splitting pass</font>
<div> </div>
</div>
<div style="overflow-wrap: break-word;">Hello,
<div><br>
</div>
<div>I’d like to give a status update to the community about the recently-added hot/cold splitting pass. I'll provide some motivation for the pass, describe its implementation, summarize recent/ongoing work, and share early results.</div>
<div><br>
</div>
<div># Motivation</div>
<div><br>
</div>
<div>We (at Apple) have found that memory pressure from resident pages of code is significant on embedded devices. In particular, this pressure spikes during app launches. We’ve been looking into ways to reduce memory pressure. Hot/cold splitting is
one part of a solution.</div>
<div><br>
</div>
<div># What does hot/cold splitting do?</div>
<div><br>
</div>
<div>
<div>The hot/cold splitting pass identifies cold basic blocks and moves them into separate functions. The linker must order newly-created cold functions away from the rest of the program (say, into a cold section). The idea here is to have these cold
pages faulted in relatively infrequently (if at all), and to improve the memory locality of code outside of the cold area.</div>
<div><br>
</div>
<div>The pass considers profile data, traps, uses of the `<span style="font-style:normal">cold</span><i>`</i> attribute, and exception-handling code to identify cold blocks. If the pass identifies a cold region that's profitable to
extract, it uses LLVM's CodeExtractor utility to split the region out of its original function. Newly-created cold functions are marked `minsize` (-Oz). The splitting process may occur multiple times per function.</div>
<div><br>
</div>
<div>The choice to perform splitting at the IR level gave us a lot of flexibility. It allowed us to quickly target different architectures and evaluate new phase orderings. It also made it easier to split out highly complex subgraphs of CFGs (with
both live-ins and live-outs). One disadvantage is that we cannot easily split out EH pads (<a href="http://llvm.org/PR39545" target="_blank">llvm.org/PR39545</a>). However, our experiments show that doing so only increases the total amount of split code by 2% across
the entire iOS shared cache.</div>
</div>
<div><br>
</div>
<div># Recent/ongoing work</div>
<div><br>
</div>
<div>Aditya and Sebastian contributed the hot/cold splitting pass in September 2018 (r341669). Since then, work on the pass has continued steadily. It gained the ability to extract larger cold regions (r345209), compile-time improvements (r351892,
r351894), and a more effective cost model (r352228). With some experimentation, we found that scheduling splitting before inlining gives better code size results without regressing memory locality (r352080). Along the way, CodeExtractor got better at handling
debug info (r344545, r346255), and a few other issues in this utility were fixed (r348205, r350420).</div>
<div><br>
</div>
<div>At this point, we're able to build & run our software stack with hot/cold splitting enabled. We’d like to introduce a CC1 option to safely toggle splitting on/off (<a href="https://reviews.llvm.org/D57265" target="_blank">https://reviews.llvm.org/D57265</a>).
That would help experiment with and/or deploy the pass.</div>
<div><br>
</div>
<div># Early results</div>
<div><br>
</div>
<div>On internal memory benchmarks, we consistently saw that code page faults were more concentrated with splitting enabled. With splitting, the set of the most-frequently-accessed 95% (99%) of code pages was 10% (resp. 3.6%) smaller. We used a facility
in the xnu VM to force pages to be faulted periodically, and ktrace, to collect this data. We settled on this approach because the alternatives (e.g. directly sampling RSS of various processes) gave unstable results, even when measures were taken to stabilize
a device (e.g. disabling dynamic frequency switching, SMP, and various other features).</div>
<div><br>
</div>
<div>On arm64, the performance impact of enabling splitting in the LLVM test suite appears to be in the noise. We think this is because split code amount to just 0.1% of all the code in the test suite. Across the iOS shared cache we see that 0.9% of
code is split, with higher percentages in key frameworks (e.g. 7% in libdispatch). For three internal benchmarks, we see geomean score improvements of 1.58%, 0.56%, and 0.27% respectively. We think these results are promising. I’d like to encourage others
to evaluate the pass and share results.</div>
<div><br>
</div>
<div>Thanks!</div>
<div><br>
</div>
<div>vedant</div>
</div>
</div>
<span style="color:rgb(255,255,255);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">_______________________________________________</span><br style="color:rgb(255,255,255);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="color:rgb(255,255,255);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">LLVM
Developers mailing list</span><br style="color:rgb(255,255,255);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="color:rgb(255,255,255);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline"><a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a></span><br style="color:rgb(255,255,255);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="color:rgb(255,255,255);font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline"><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></span></div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><span style="font-family:Times;font-size:medium"><table cellspacing="0" cellpadding="0"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small"><td nowrap style="border-top:2px solid rgb(213,15,37)">Teresa Johnson |</td><td nowrap style="border-top:2px solid rgb(51,105,232)"> Software Engineer |</td><td nowrap style="border-top:2px solid rgb(0,153,57)"> <a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a> |</td><td nowrap style="border-top:2px solid rgb(238,178,17)"><br></td></tr></tbody></table></span></div></div></div></div>