<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Alina & Arthur,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I've investigated the performance impact for us and can now say a little bit more now where we are. <span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; background: var(--white);">Switching now would lead to
performance regressions (*) in the initial set of 4 benchmarks we care about. One benchmark is overall neutral but shows regressions where we don't really want them. Hopefully, your fix for PR<a href="https://bugs.llvm.org/show_bug.cgi?id=48715" id="LPlnk187192">48715</a> that
I raised today will solve that, many thanks for the amazingly speedy reply! A second benchmark is a bit of disaster, still need to look into that, and a third shows a relatively small regression but it's significant for that benchmark and am looking into that
now, and need to look into the fourth benchmark.</span></div>
<div id="appendonsend"></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Code-generation is *very* different for the cases I am look at and I am profiling and analysing things, for which I need some time. This leads to my question: can you remind me about the timelines? I hope we can work in tandem to have at least the major issues
resolved before we switch.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thanks for working on this, and for your help and speedy replies,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Sjoerd.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
(*) I am mostly looking at smaller cores, and very tight loops (baremetal) and guess that most people look at bigger cores where some of these codegen differences have no or a different impact.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Sjoerd Meijer <Sjoerd.Meijer@arm.com><br>
<b>Sent:</b> 10 January 2021 11:36<br>
<b>To:</b> Alina Sbirlea <alina.sbirlea@gmail.com>; llvm-dev <llvm-dev@lists.llvm.org><br>
<b>Cc:</b> Arthur Eubanks <aeubanks@google.com>; Sharma, Reshabh Kumar <Reshabhkumar.Sharma@amd.com>; Matt Arsenault <arsenm2@gmail.com>; nhaehnle@gmail.com <nhaehnle@gmail.com>; Philip Reames <listmail@philipreames.com>; Chen, Yuanfang <Yuanfang.Chen@sony.com><br>
<b>Subject:</b> Re: [llvm-dev] New pass manager for optimization pipeline status and questions</font>
<div> </div>
</div>
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hello,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div><span style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt">Our main showstop</span><span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">per PR46858 (code-size) has been resolved.
Many thanks to Arthur for fixing this.</span></div>
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)"><br>
</span></div>
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">We have seen performance problems too, but first needed to port things to the NPM to get a better picture of this, which is what we've done. Now I'd need to
reassess where we these performance problems, and hope to do that before the end of this week.</span></div>
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)"><br>
</span></div>
<div><span style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">Cheers,<br>
Sjoerd.</span></div>
<div id="x_appendonsend"></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Alina Sbirlea <alina.sbirlea@gmail.com><br>
<b>Sent:</b> 08 January 2021 23:55<br>
<b>To:</b> llvm-dev <llvm-dev@lists.llvm.org><br>
<b>Cc:</b> Arthur Eubanks <aeubanks@google.com>; Sharma, Reshabh Kumar <Reshabhkumar.Sharma@amd.com>; Matt Arsenault <arsenm2@gmail.com>; nhaehnle@gmail.com <nhaehnle@gmail.com>; Sjoerd Meijer <Sjoerd.Meijer@arm.com>; Philip Reames <listmail@philipreames.com>;
Chen, Yuanfang <Yuanfang.Chen@sony.com><br>
<b>Subject:</b> Re: [llvm-dev] New pass manager for optimization pipeline status and questions</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Hello,</div>
<div><br>
</div>
<div>Reviving this thread as the switch to the new pass manager is getting very close now.</div>
<div><br>
</div>
<div>Arthur has been resolving most of the issues, and the remaining ones are tracked under the umbrella bug here:
<a href="https://bugs.llvm.org/show_bug.cgi?id=46649">https://bugs.llvm.org/show_bug.cgi?id=46649</a>.</div>
<div>Regarding the opt failures, the two remaining ones (AMD related) were posted in this discussion: <a href="https://lists.llvm.org/pipermail/llvm-dev/2020-December/147130.html">https://lists.llvm.org/pipermail/llvm-dev/2020-December/147130.html</a>.</div>
<div><br>
</div>
<div>Thank you to everyone who has done offline testing, opening bugs to track and sending and reviewing patches to move forward the switch.</div>
<div>With the switch getting so close, it would help have a smooth transition if any new issues you encounter were filed under the umbrella bug now.</div>
<div><br>
</div>
<div>Thank you,</div>
<div>Alina</div>
<div><br>
</div>
</div>
<br>
<div class="x_x_gmail_quote">
<div dir="ltr" class="x_x_gmail_attr">On Wed, Jul 29, 2020 at 10:13 PM Alina Sbirlea via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br>
</div>
<blockquote class="x_x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div class="x_x_gmail_quote">
<div dir="ltr" class="x_x_gmail_attr">On Wed, Jul 29, 2020 at 3:37 AM Sjoerd Meijer <<a href="mailto:Sjoerd.Meijer@arm.com" target="_blank">Sjoerd.Meijer@arm.com</a>> wrote:<br>
</div>
<blockquote class="x_x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="margin:0px; font-size:15px; background-color:rgb(255,255,255)">Hi,</div>
<div style="margin:0px; font-size:15px; background-color:rgb(255,255,255)"><br>
</div>
<div style="margin:0px; font-size:15px; background-color:rgb(255,255,255)">
<div style="margin:0px; background-color:rgb(255,255,255)">> Yes, I think big performance and code-size regressions need to be investigated. It will be hard to quantify what "big" is though, but we should definitely track such regressions.</div>
<div style="margin:0px; background-color:rgb(255,255,255)">> Due to the time-boxed approach we may need to discuss moving forward with the switch with some regressions deemed acceptable, but those considered blockers should be prioritized of course.</div>
<br>
</div>
<div style="margin:0px; font-size:15px; background-color:rgb(255,255,255)">Agreed, SGTM.</div>
<div style="margin:0px; font-size:15px; background-color:rgb(255,255,255)"><br>
</div>
<div style="margin:0px; font-size:15px; background-color:rgb(255,255,255)">> I think it's reasonable to add a firm switch before clang-13 and before the end of this year, with intermediary milestones (e.g. filing blockers for user regressions in the next 1-2
months).</div>
<div style="margin:0px; font-size:15px; background-color:rgb(255,255,255)">> I'm inclined to favor a tighter deadline, the motivation here being to ensure that working on potential blockers is prioritized with plenty of time to spare, so the switch remains
time-boxed.</div>
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I am not entirely sure yet about the tighter deadline, but I am optimistic things might start falling into place soon. Let me explain.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I think you've noticed the code-size issue that I raised. Switching before that being fixed would be a real non-starter for us, but if there's consensus the community want to switch then we'll deal with that of course.</div>
</div>
</blockquote>
<blockquote class="x_x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Because (some) of these regression are so big, I am hoping we are missing something obvious and that with a few things fixed we solve most of these cases, which is why I am optimistic, and also because of next point:</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
After code-size, I'm now looking at performance, and things look a lot better there for us. I do see some up and down behaviour: some wins in some benchmarks overall, but also some regressions. I will need to do some more homework here, as I need to check if
small downstream divergence play a role here, and need to eliminate that. In some benchmarks where we win overall, there are a few cases that we really don't want to regress, so I need to look into this. I think these are problems that we need to fix, but
I will start raising issues for them just for visibility. Overall, there are less to none non-starters in this area I think.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
But because these are time-consuming exercises, and it's the summer months, I am getting slightly nervous about the tight(er) deadline. :-)</div>
</div>
</blockquote>
<div><br>
</div>
<div>I think it's great to make it clear which items are critical for you, and having those as blocker makes perfect sense. The code-size issue you added to the umbrella bug makes sense to me, thank you for filing that. I think it's also fair to ask for reasonable
delays or help addressing such issues.</div>
<div><br>
</div>
<div>I completely agree that discovering and addressing these is a time-consuming endeavour, so I think there's some room for flexibility here. Again, the main reason I'm advocating for a preliminary tighter deadline is so we can ensure prioritization of discovering
and addressing such blockers early. As long as the extended deadline (end of this year and clang-13) remains firm I think we'd be in a good position.</div>
<div>A tighter deadline is useful as a checkpoint with the community of: "who is regressed if we switched today", "are the regressions at this point acceptable to make the switch", "what is the status of the investigations into these regressions", "what resources
are needed to make progress in the remaining blockers". I propose having the tighter deadline half-way through (middle of October) and work towards evaluating the status of the switch then and decide on extensions as needed. Does this sound like a reasonable
plan?</div>
<div><br>
</div>
<div>The end goal is for the flag flip to be an overall benefit, not to cause disruptions :-).</div>
<div><br>
</div>
<div>Best,</div>
<div>Alina</div>
<div> </div>
<blockquote class="x_x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span>Cheers,</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span>Sjoerd.</span></div>
</div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>I think the progress so far looks great</div>
<div> </div>
<blockquote class="x_x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div id="x_x_gmail-m_7614180304928163968gmail-m_-1666576133421941422appendonsend">
</div>
<hr style="display:inline-block; width:98%">
<div id="x_x_gmail-m_7614180304928163968gmail-m_-1666576133421941422divRplyFwdMsg" dir="ltr">
<font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Alina Sbirlea <<a href="mailto:asbirlea@google.com" target="_blank">asbirlea@google.com</a>><br>
<b>Sent:</b> 28 July 2020 18:23<br>
<b>To:</b> Sjoerd Meijer <<a href="mailto:Sjoerd.Meijer@arm.com" target="_blank">Sjoerd.Meijer@arm.com</a>><br>
<b>Cc:</b> Philip Reames <<a href="mailto:listmail@philipreames.com" target="_blank">listmail@philipreames.com</a>>; Chandler Carruth <<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>>; Eric Christopher <<a href="mailto:echristo@gmail.com" target="_blank">echristo@gmail.com</a>>;
llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] New pass manager for optimization pipeline status and questions</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div>
<div dir="ltr">On Fri, Jul 24, 2020 at 12:54 PM Sjoerd Meijer <<a href="mailto:Sjoerd.Meijer@arm.com" target="_blank">Sjoerd.Meijer@arm.com</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi Alina,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I think this is an excellent direction, this is the direction we should take here. <span style="background-color:rgba(0,0,0,0); color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt">Just a somewhat irrelevant disagreement on this
though:</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(32,31,30); font-size:15px; background-color:rgb(255,255,255); display:inline"><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgba(0,0,0,0)">> </span><span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; background-color:rgba(0,0,0,0); display:inline">Philip's
point is spot on that we are deficient now in the testing of the LegacyPassManager,</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(32,31,30); font-size:15px; background-color:rgb(255,255,255); display:inline"><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; background-color:rgba(0,0,0,0); display:inline">I disagree because the LPM is still the default and I appreciated Hans' reply: "</span><span style="color:rgb(32,31,30); font-size:15px; background-color:rgb(255,255,255); display:inline"><span style="font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; background-color:rgba(0,0,0,0); color:rgb(0,0,0); display:inline">Defaults </span><span style="font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; background-color:rgba(0,0,0,0); color:rgb(0,0,0); display:inline">tend
to be popular</span></span><span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; background-color:rgba(0,0,0,0); display:inline">". But <span style="background-color:rgb(255,255,255); display:inline">this is the direction
I like:</span></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="margin:0px">> This means prioritizing those blockers over other LLVM work. The current umbrella bug is <a href="https://bugs.llvm.org/show_bug.cgi?id=46649" target="_blank" style="margin:0px">PR46649</a>.</div>
<div style="margin:0px"><br style="color:rgb(50,49,48); background-color:rgb(255,255,255)">
</div>
Just checking: do you accept both performance and code-size regressions as blockers here?</div>
</div>
</blockquote>
<div><br>
</div>
<div>Yes, I think big performance and code-size regressions need to be investigated. It will be hard to quantify what "big" is though, but we should definitely track such regressions.</div>
<div>Due to the time-boxed approach we may need to discuss moving forward with the switch with some regressions deemed acceptable, but those considered blockers should be prioritized of course.</div>
<div> </div>
<blockquote style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(32,31,30); font-size:15px; background-color:rgb(255,255,255); display:inline"><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(32,31,30); font-size:15px; background-color:rgb(255,255,255); display:inline"><span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; background-color:rgb(255,255,255); display:inline">> My point
is that we want and should work with users to make the transition smooth, but we do very much need user (meaning companies using LLVM) involvement here in order to not delay the switch further.</span><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(32,31,30); font-size:15px; background-color:rgb(255,255,255); display:inline"><span style="color:rgb(50,49,48); font-size:16px; background-color:rgb(255,255,255); display:inline"><br>
</span></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(32,31,30); font-size:15px; background-color:rgb(255,255,255); display:inline"><span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; background-color:rgb(255,255,255); display:inline">That's clear,
and agreed. </span></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(32,31,30); font-size:15px; background-color:rgb(255,255,255); display:inline"><span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt; background-color:rgb(255,255,255); display:inline">I would like
to remark here that currently, when a commit regresses one benchmark that is important for someone, that is enough justification most of the time for a revert of that commit. That's why I </span></span><span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt">surprised
that it looked like we were not setting code-quality goals and requirements before switching. And </span><span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt">what I would like to ask here is to provide reasonable enough
time for people to look into switching to the NPM, to evaluate this, and then file bugs under PR46649. Just collecting data, evaluating problems, filings bugs can already time-consuming, and then I guess they need fixing too. This also needs to fit in people's
plans right now.</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(0,0,0); font-family:calibri,arial,helvetica,sans-serif; font-size:12pt"><br>
</span></div>
<div style="font-size:12pt; color:rgb(0,0,0)"><font face="calibri, arial, helvetica, sans-serif">But it sounds reasonable to me that this is time-boxed. </font><span style="font-family:calibri,arial,helvetica,sans-serif; color:rgb(0,0,0); font-size:12pt">Given
that switching is quite some work I think, switching before the clang-12 release would be unreasonable, and if clang-13 is in half a year from now, that already sounds perhaps somewhat reasonable, but might be tight. </span></div>
</div>
</blockquote>
<div><br>
</div>
<div>I think it's reasonable to add a firm switch before clang-13 and before the end of this year, with intermediary milestones (e.g. filing blockers for user regressions in the next 1-2 months).</div>
<div>I'm inclined to favor a tighter deadline, the motivation here being to ensure that working on potential blockers is prioritized with plenty of time to spare, so the switch remains time-boxed.</div>
<div><br>
</div>
<div>Best,</div>
<div>Alina</div>
<div> </div>
<blockquote style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-size:12pt; color:rgb(0,0,0)"><span style="font-family:calibri,arial,helvetica,sans-serif; color:rgb(0,0,0); font-size:12pt"></span></div>
<div style="font-size:12pt; color:rgb(0,0,0)"><span style="font-family:calibri,arial,helvetica,sans-serif; color:rgb(0,0,0); font-size:12pt">Thanks,</span></div>
<div style="font-size:12pt; color:rgb(0,0,0)"><span style="font-family:calibri,arial,helvetica,sans-serif; color:rgb(0,0,0); font-size:12pt">Sjoerd.</span></div>
<div id="x_x_gmail-m_7614180304928163968gmail-m_-1666576133421941422x_gmail-m_-7395407641225908880appendonsend">
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr style="display:inline-block; width:98%">
<div id="x_x_gmail-m_7614180304928163968gmail-m_-1666576133421941422x_gmail-m_-7395407641225908880divRplyFwdMsg" dir="ltr">
<font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Alina Sbirlea <<a href="mailto:asbirlea@google.com" target="_blank">asbirlea@google.com</a>><br>
<b>Sent:</b> 24 July 2020 19:51<br>
<b>To:</b> Sjoerd Meijer <<a href="mailto:Sjoerd.Meijer@arm.com" target="_blank">Sjoerd.Meijer@arm.com</a>><br>
<b>Cc:</b> Philip Reames <<a href="mailto:listmail@philipreames.com" target="_blank">listmail@philipreames.com</a>>; Chandler Carruth <<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>>; Eric Christopher <<a href="mailto:echristo@gmail.com" target="_blank">echristo@gmail.com</a>>;
llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] New pass manager for optimization pipeline status and questions</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi all,
<div><br>
</div>
<div>The current plan is to prioritize enabling the NPM as soon as possible, and that includes addressing any blockers that are known or arise. This means prioritizing those blockers over other LLVM work. The current umbrella bug is <a href="https://bugs.llvm.org/show_bug.cgi?id=46649" target="_blank">PR46649</a>.</div>
<div><br>
</div>
<div>Philip's point is spot on that we are deficient now in the testing of the LegacyPassManager, because so many have already made the switch (FWIW Google switched more than 2 years ago).</div>
<div><br>
</div>
<div>It's not constructive for the LLVM community to just flip the switch and break current LPM users. The purpose of these communications to llvm-dev and the bug tracking is to be informative as to the planned direction and make as quick of a progress as possible.</div>
<div>Please keep in mind that the work on the NPM has been going on for many years and many customers have switched years ago, and delaying this for even an additional year is not acceptable for the code health and stability of LLVM.<br>
</div>
<div><br>
</div>
<div>My point is that we want and should work with users to make the transition smooth, but we do very much need user (meaning companies using LLVM) involvement here in order to not delay the switch further.</div>
<div><br>
</div>
<div>Best,</div>
<div>Alina</div>
<div><br>
</div>
</div>
<br>
<div>
<div dir="ltr">On Thu, Jul 23, 2020 at 2:59 AM Sjoerd Meijer <<a href="mailto:Sjoerd.Meijer@arm.com" target="_blank">Sjoerd.Meijer@arm.com</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I am not in favour of just flipping the switch and then deal with all the fall-out, because <span style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt">we see major regressions that would be unacceptable for our users. Thus,
not only would this be very disruptive, also our releases are based on a certain trunk versions, so we would need to revert back to the legacy PM downstream and thus diverge from upstream which wouldn't be ideal for us. About the regressions, s</span><span style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt">ee
the message/thread that I kicked off earlier (</span><a href="http://lists.llvm.org/pipermail/llvm-dev/2020-July/143646.html" target="_blank" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt">http://lists.llvm.org/pipermail/llvm-dev/2020-July/143646.html</a><span style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt">)
which was quickly followed up by this thread. </span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt"><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I would like to see here if we are interesting in defining a few criteria that must be met before we switch:</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<ol>
<li>Correctness, which obviously always must come first: looks like this is covered by bots that are running with the NPM, and by downstream users. From the latest messages, I am getting we are there, or nearly there.</li><li>Performance (i.e. optimising for speed), </li><li>Code-size.</li></ol>
<div>With 1) correctness box covered and ticked, is now the time to look at codegen quality: 2) performance and 3) code-size? Would it be reasonable that we create a plan or timeline to address this, and thus allow time that these issues can be addressed?</div>
<div><br>
</div>
<div>We are now ready to start tuning the NPM for code-size. Perhaps we are late to the NPM party (but that was a priority and bandwidth issue), but perhaps with correctness fixed this is actually the right time. I only ran numbers for code-size, and haven't
even looked at performance numbers yet, which we would also need to do and takes time.</div>
<div><br>
</div>
<div>Cheers,</div>
<div>Sjoerd.</div>
<div><br>
</div>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div id="x_x_gmail-m_7614180304928163968gmail-m_-1666576133421941422x_gmail-m_-7395407641225908880x_gmail-m_-1631098210803735589appendonsend">
</div>
<hr style="display:inline-block; width:98%">
<div id="x_x_gmail-m_7614180304928163968gmail-m_-1666576133421941422x_gmail-m_-7395407641225908880x_gmail-m_-1631098210803735589divRplyFwdMsg" dir="ltr">
<font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists.llvm.org</a>> on behalf of Eric Christopher via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Sent:</b> 23 July 2020 02:05<br>
<b>To:</b> Philip Reames <<a href="mailto:listmail@philipreames.com" target="_blank">listmail@philipreames.com</a>>; Alina Sbirlea <<a href="mailto:asbirlea@google.com" target="_blank">asbirlea@google.com</a>>; Chandler Carruth <<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>><br>
<b>Cc:</b> llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] New pass manager for optimization pipeline status and questions</font>
<div> </div>
</div>
<div>
<div dir="ltr">FWIW I'm in favor of this direction while making sure that we keep focus on removing the vestiges of the old pass manager for the code health impact to the project.
<div><br>
</div>
<div>-eric</div>
</div>
<br>
<div>
<div dir="ltr">On Wed, Jul 22, 2020 at 3:15 PM Philip Reames via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div>
<p style="margin-top: 0px; margin-bottom: 0px;">(I'm probably going to derail your thread, sorry about that.)</p>
<p style="margin-top: 0px; margin-bottom: 0px;">I think at this point, we should just bite the bullet and make the switch to NPM by default for Clang's optimization pipeline. Today.</p>
<p style="margin-top: 0px; margin-bottom: 0px;">Why? Because many of our downstream consumers have already switched. Google has. We (Azul) have. I think I've heard the same for a couple other major contributors. Why does this matter? Testing. At the
current moment, the vast majority of testing the project gets is exercising NPM, not LPM.</p>
<p style="margin-top: 0px; margin-bottom: 0px;">NPM is functionally complete for Clang optimization. There might be a few missing cases around the sanitizers, but last I heard those were on the edge of being fixed.</p>
<p style="margin-top: 0px; margin-bottom: 0px;">I think we should make the switch, and deal with any fall out as regressions. If we made the change immediately after a release branch, we'd have several months to address any major issues before the next release.
<br>
</p>
<p style="margin-top: 0px; margin-bottom: 0px;">Philip<br>
</p>
<div>On 7/22/20 2:39 PM, Arthur Eubanks via llvm-dev wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hi all,</div>
<div><br>
</div>
<div>I wanted to give a quick update on the status of NPM for the IR optimization pipeline and ask some questions.</div>
<div><br>
</div>
<div>In the past I believe there were thoughts that NPM was basically ready because all of check-llvm and check-clang passed when -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON was specified. But that CMake flag did not apply to opt and any tests running something
like `opt -foo-pass -bar-pass` (which is the vast majority of check-llvm tests) were still using the legacy PM. The intended way to use NPM was to use the -passes flag, e.g. `opt -passes='foo,bar'`.</div>
<div><br>
</div>
<div>I've added a -enable-new-pm flag to opt to force running NPM passes even when `opt -foo-pass` is used. This is because I didn't want to go through every single test and figure out which ones should be using both -foo-pass and -passes=foo. Switching on
-enable-new-pm currently leads to ~1800 check-llvm failures. I've documented the failed tests count per directory in <a href="https://bugs.llvm.org/show_bug.cgi?id=46651" target="_blank">https://bugs.llvm.org/show_bug.cgi?id=46651</a> (some have been fixed
since that was posted).</div>
<div><br>
</div>
<div>This has led to real bugs in NPM being discovered and fixed (e.g. some optnone issues).</div>
<div><br>
</div>
<div>But a large portion of the remaining failures are because codegen-only passes haven't been ported to NPM yet. That's fine for the optimization pipeline NPM transition since it doesn't affect the optimization pipeline, but it does present an issue with
the approach of the -enable-new-pm flag (which would by default become true alongside the NPM transition). Lots of tests are testing codegen-specific passes via opt (e.g. `opt -amdgpu-lower-intrinsics`) and they can't use NPM (yet).</div>
<div><br>
</div>
<div>I was thinking either we have a way of identifying codegen-only passes and revert back to the legacy PM in opt whenever we see one, or we go back to considering the originally intended approach of adding an equivalent `-passes=` RUN to all tests that should
be also running under NPM.</div>
<div><br>
</div>
<div>I'm not sure of a nice and clean solution to identify codegen-only passes. We could go and update every instance of INITIALIZE_PASS to take another parameter indicating if it's codegen-only. Or we could just have a central list somewhere where we check
if the pass is in some hardcoded list or has some prefix (e.g. "x86-").</div>
<div><br>
</div>
<div>The approach of adding equivalent `-passes=` RUN lines to all relevant tests seems daunting, but not exactly sure how daunting. Maybe it's possible to script something and see what fails? We'd still need some way to identify codegen-only passes to make
sure we don't miss anything, and we'd need to distinguish between analyses and normal passes. Also, it would slow down test execution since we'd run a lot more tests twice, but maybe that's not such a big deal? Maybe it's good to have most tests running against
the legacy PM even when NPM is on by default?</div>
<div><br>
</div>
<div>Thoughts?</div>
<div><br>
</div>
This is split off from <a href="http://lists.llvm.org/pipermail/llvm-dev/2020-July/143395.html" target="_blank">http://lists.llvm.org/pipermail/llvm-dev/2020-July/143395.html</a>.
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
LLVM Developers mailing list
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote>
</div>
</div>
</div>
</body>
</html>