[llvm-dev] [RFC] Deprecating the legacy pass manager for the optimization pipeline

Wed Aug 25 09:38:03 PDT 2021

Legacy pass manager is a very large piece of infrastructure in LLVM, and for that reason I think it's especially important to provide a good long deprecation period. Up until now, there's been no deadline for its removal, so for us to say "in 2 months passes are going to start getting deleted" is unfair. As it stands, NPM is a many-years-long project, and it's always been the case that it's going to replace LPM "soon", but there's never been a date. (if I'm wrong, please correct me) Given the level of effort required to migrate to new pass manager, it no surprise that users may not have done it until "soon" became "on this specific date". I know that "downstreams are on their own" is our policy, but it's also the case that we value not making things difficult for downstreams. I know I've repeatedly been told to back out changes due to push back from downstreams.

I don't think a 6 month deprecation period is a big ask. This RFC is essentially proposing "let's remove some dead code", so I don't see how waiting is going to block anybody's work. And as Björn mentioned in his other email (that I assume he meant to CC the lists on, but I'll refrain from quoting him verbatim), there are things we can get started on in the meantime, such as cleaning up the lit tests and changing CMake defaults. But please, let's not remove any actual code until we've had a formal deprecation period.

If the concern is that people will keep pushing code that uses LPM infrastructure, then we can mark it deprecated so the build bots reject it. Preferably we'd do some sort of `#define LLVM_NPM_ATTRIBUTE_DEPRECATED(decl, msg) LLVM_ATTRIBUTE_DEPRECATED(decl, msg)` thing so that downstreams can just redefine one thing instead of deleting a bunch of deprecation flags.

Thanks,
   Chris Tetreault

-----Original Message-----
From: Fāng-ruì Sòng <maskray at google.com> 
Sent: Tuesday, August 24, 2021 4:53 PM
To: Chris Tetreault <ctetreau at quicinc.com>
Cc: Arthur Eubanks <aeubanks at google.com>; llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for the optimization pipeline

WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

On 2021-08-24, Chris Tetreault wrote:
>Let me clarify that when I say “it’s fine to begin removing LPM in LLVM 14”, I mean that it’s fine to begin removing LPM once LLVM 14 is released, and the version string in main is set to 15.

While I think such a deprecation and removal policy may be fine for many non-trivial things, asking this may be too much for the pass manager. With a grain of salt, "downstreams are on their own." New PM migration and legacy PM removal has been repeatedly forewarned.  I think llvm-project has done above and beyond what it was expected.  The flip in git was done in February.

After 13.0.0 is released for, say 2 months, which additional attests its robustness, I think removing non-codegen legacy PM pieces should be fine.

>From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Chris 
>Tetreault via llvm-dev
>Sent: Tuesday, August 24, 2021 3:39 PM
>To: Arthur Eubanks <aeubanks at google.com>; Fāng-ruì Sòng 
><maskray at google.com>
>Cc: llvm-dev at lists.llvm.org
>Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for 
>the optimization pipeline
>
>
>WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>Downstreams could be re-enabling the legacy pass manager in CMake. It’s still there, and still supported. If a downstream has made non-trivial changes to the pass pipeline for legacy pass manager, it’s going to be a ton of work to ensure performance parity when enabling new pass manager.
>
>I think, if LLVM is built with LLVM_ENABLE_NEW_PASS_MANAGER set to OFF, then you should get a big scary warning at CMake configure time, but clang/opt should not complain. If LLVM is built with DLLVM_ENABLE_NEW_PASS_MANAGER set to ON, but the flag is passed to clang/opt to build using the legacy pass manager, you should get a warning from clang/opt.
>
>In my opinion, if these deprecation warnings make it in for LLVM 13, then it’s fine to begin removing LPM in LLVM 14.
>
>thanks,
>   Chris Tetreault
>
>From: Arthur Eubanks <aeubanks at google.com<mailto:aeubanks at google.com>>
>Sent: Tuesday, August 24, 2021 1:47 PM
>To: Fāng-ruì Sòng <maskray at google.com<mailto:maskray at google.com>>
>Cc: Chris Tetreault 
><ctetreau at quicinc.com<mailto:ctetreau at quicinc.com>>; 
>llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>; Tom Stellard 
><tstellar at redhat.com<mailto:tstellar at redhat.com>>
>Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for 
>the optimization pipeline
>
>
>WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>If nobody is setting -DLLVM_ENABLE_NEW_PASS_MANAGER=off then I'm not sure that we need to go through the whole deprecation dance. Is there anybody doing that?
>
>But if we do the deprecation dance, should it be a CMake warning or a clang warning? I'm not sure people will notice a CMake warning.
>And would putting that in the 13.x branch be good enough to start removing after 14.x?
>
>On Tue, Aug 24, 2021 at 12:26 PM Fāng-ruì Sòng <maskray at google.com<mailto:maskray at google.com>> wrote:
>On Tue, Aug 24, 2021 at 12:21 PM Chris Tetreault <ctetreau at quicinc.com<mailto:ctetreau at quicinc.com>> wrote:
>>
>> Since deprecating something is giving warning that it’s going to be removed, I don’t think it’s fair to deprecate and remove in one step. If legacy pass manager isn’t currently formally deprecated (as in, loudly complains when you try to use it), then I’d personally like to see it deprecated for a release before any steps to completely remove it are taken.
>
>I just wanted to say the same thing:)
>
>I don't mind that we install a warning for 
>-DLLVM_ENABLE_NEW_PASS_MANAGER=off
>in the release/13.x branch so that the signal is clearer to downstream users.
>
>> From: llvm-dev 
>> <llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces at lists.llvm.o
>> rg>> On Behalf Of Arthur Eubanks via llvm-dev
>> Sent: Tuesday, August 24, 2021 12:10 PM
>> To: Fangrui Song <maskray at google.com<mailto:maskray at google.com>>
>> Cc: llvm-dev 
>> <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
>> Subject: Re: [llvm-dev] [RFC] Deprecating the legacy pass manager for 
>> the optimization pipeline
>>
>>
>>
>> WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>>
>> I probably should have said "deprecating and removing".
>>
>>
>>
>> My intention was to remove it completely. -DLLVM_ENABLE_NEW_PASS_MANAGER wouldn't do anything and we'd remove the -flegacy-pass-manaager/-fexperimental-new-pass-manager flags, as well as the corresponding lld flags.
>>
>>
>>
>> On Tue, Aug 24, 2021 at 12:02 PM Fangrui Song <maskray at google.com<mailto:maskray at google.com>> wrote:
>>
>> On 2021-08-24, Arthur Eubanks via llvm-dev wrote:
>> >The new pass manager has been on by default since the 13 branch. Now 
>> >that we've branched for 14, I'd like to start the process of 
>> >deprecating and removing legacy pass manager support for the 
>> >optimization pipeline. This includes clang, opt, and lld LTO support.
>> >
>> >Note that this doesn't apply to the codegen pipeline since there's 
>> >no new pass manager support for that yet.
>> >
>> >Are there any objections?
>>
>> "deprecating" and 'removing" are different.
>>
>> For 14.0.0, do you plan that -DLLVM_ENABLE_NEW_PASS_MANAGER=off will 
>> give a warning or will be completely unsupported?
>
>
>
>--
>宋方睿