<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
font-variant:normal !important;
color:black;
text-transform:none;
font-emphasize:none;
font-weight:normal;
font-style:normal;
text-decoration:none none;
vertical-align:baseline;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><b>From:</b> Arthur Eubanks <aeubanks@google.com>
<br>
<b>Sent:</b> Thursday, July 23, 2020 10:58 AM<br>
<b>To:</b> Chen, Yuanfang <Yuanfang.Chen@sony.com><br>
<b>Cc:</b> LLVM Developers' List <llvm-dev@lists.llvm.org><br>
<b>Subject:</b> Re: [llvm-dev] New pass manager for optimization pipeline status and questions<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in">On Wed, Jul 22, 2020 at 4:05 PM Chen, Yuanfang <<a href="mailto:Yuanfang.Chen@sony.com" target="_blank">Yuanfang.Chen@sony.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<b>From:</b> llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists.llvm.org</a>>
<b>On Behalf Of </b>Arthur Eubanks via llvm-dev<br>
<b>Sent:</b> Wednesday, July 22, 2020 2:39 PM<br>
<b>To:</b> llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> [llvm-dev] New pass manager for optimization pipeline status and questions<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
Hi all,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
I wanted to give a quick update on the status of NPM for the IR optimization pipeline and ask some questions.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
In the past I believe there were thoughts that NPM was basically ready because all of check-llvm and check-clang passed when -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON was specified. But that CMake flag did not apply to opt and any tests running something like
`opt -foo-pass -bar-pass` (which is the vast majority of check-llvm tests) were still using the legacy PM. The intended way to use NPM was to use the -passes flag, e.g. `opt -passes='foo,bar'`.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
I've added a -enable-new-pm flag to opt to force running NPM passes even when `opt -foo-pass` is used. This is because I didn't want to go through every single test and figure out which ones should be using both -foo-pass and -passes=foo. Switching on -enable-new-pm
currently leads to ~1800 check-llvm failures. I've documented the failed tests count per directory in <a href="https://bugs.llvm.org/show_bug.cgi?id=46651" target="_blank">https://bugs.llvm.org/show_bug.cgi?id=46651</a> (some have been fixed since that was
posted).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
This has led to real bugs in NPM being discovered and fixed (e.g. some optnone issues).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
But a large portion of the remaining failures are because codegen-only passes haven't been ported to NPM yet. That's fine for the optimization pipeline NPM transition since it doesn't affect the optimization pipeline, but it does present an issue with the approach
of the -enable-new-pm flag (which would by default become true alongside the NPM transition). Lots of tests are testing codegen-specific passes via opt (e.g. `opt -amdgpu-lower-intrinsics`) and they can't use NPM (yet).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black">I think the ideal way is just to port these to NPM. The problem is if the opt pipeline NPM switch is blocked on this, we’re forcing the targets to start porting which I’m not sure if target owners want to do.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black">On the other end, we’re (almost) actually ready to make these target IR passes use NPM for testing purpose with `opt` tool. Except there needs a way to expose these passes through llvm/lib/Target, llvm/lib/CodeGen rather than llvm/lib/Passes
to `opt`. As part of the codegen using NPM work, this is almost done.</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in">Grepping for INITIALIZE_PASS in llvm/lib/Target, there are 220 files with that. That means there are ~200 target specific passes. There's no way we'll be able to port all of those in a reasonable amount of time.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Plus it's not a blocker if we can detect passes are target specific and simply revert to the legacy PM in opt. Then as codegen passes are ported to NPM we can start running those on NPM instead reverting back to
the legacy PM. The exact mechanism for this TBD, although probably something similar to below.<o:p></o:p></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">220 should include machine passes. If you grep “public FunctionPass/ModulePass etc.”, we should be looking at ~60 and not every each of them is registered (tested by `opt`). A lot of them are from AMDGPU. Other
than AMDGPU, each target has 2 or 3 on average. If AMDGPU is willing to actively be involved in this and the other targets could migrate theirs, this could not only helps opt pipeline switch but also the codegen pipeline switch. Again, I meant to say these
are all “ideal” situation. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
I was thinking either we have a way of identifying codegen-only passes and revert back to the legacy PM in opt whenever we see one, or we go back to considering the originally intended approach of adding an equivalent `-passes=` RUN to all tests that should
be also running under NPM.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<span style="color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black">I would prefer the former since it sounds less pervasive.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
I'm not sure of a nice and clean solution to identify codegen-only passes. We could go and update every instance of INITIALIZE_PASS to take another parameter indicating if it's codegen-only. Or we could just have a central list somewhere where we check if the
pass is in some hardcoded list or has some prefix (e.g. "x86-").<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black">The latter seems in line with the progress has been made on codegen using NPM work where each target would maintain their own pass registry like PassRegister.def. For x86, it is (tentatively) called X86PassRegistry.def. (</span><a href="https://reviews.llvm.org/D83613" target="_blank">https://reviews.llvm.org/D83613</a>).
If It is useful for the opt pipeline NPM switch, we could find a way to factor it out.<o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in">If we can add an equivalent of something like PassBuilder::isAAPassName() to
<a href="https://reviews.llvm.org/D83613" target="_blank">https://reviews.llvm.org/D83613</a> (X86CodeGenPassBuilder::isX86CodeGenPassName()?) that would definitely work.<o:p></o:p></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">That should be straightforward. I’ll see if I could split it out from the patch.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
The approach of adding equivalent `-passes=` RUN lines to all relevant tests seems daunting, but not exactly sure how daunting. Maybe it's possible to script something and see what fails? We'd still need some way to identify codegen-only passes to make sure
we don't miss anything, and we'd need to distinguish between analyses and normal passes. Also, it would slow down test execution since we'd run a lot more tests twice, but maybe that's not such a big deal? Maybe it's good to have most tests running against
the legacy PM even when NPM is on by default?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
Thoughts?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
This is split off from <a href="http://lists.llvm.org/pipermail/llvm-dev/2020-July/143395.html" target="_blank">http://lists.llvm.org/pipermail/llvm-dev/2020-July/143395.html</a>.<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in">
I can't speak for other targets but I'd love to get AMDGPU passes converted to the NPM. Is there a howto somewhere?<o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in">I don't think so, but it's usually fairly straightforward, looking at other passes that exist in both PMs is usually good. I should rewrite <a href="https://llvm.org/docs/WritingAnLLVMPass.html" target="_blank">https://llvm.org/docs/WritingAnLLVMPass.html</a>
to reference NPM.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">But specifically for codegen passes we're currently blocked on ychen's work for codegen passes infra in NPM:
<a href="http://lists.llvm.org/pipermail/llvm-dev/2020-July/143309.html" target="_blank">
http://lists.llvm.org/pipermail/llvm-dev/2020-July/143309.html</a>.<o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>