[llvm-dev] status of CodeGen in new Pass Manager

Mon Nov 15 16:57:32 PST 2021

On Mon, Nov 15, 2021 at 2:27 PM Arthur Eubanks <aeubanks at google.com> wrote:

> `llc -O3` does not run the optimization pipeline on the IR, so IR-level
> optimizations aren't being run unless you use `opt -O3`.
> `opt -O3` optimizes the IR.
> `llc -O3` (mostly) enables MIR optimizations and better isel. But if the
> input IR isn't optimized then you lose most optimization opportunities.
> So a typical `clang -O3` would be somewhat equivalent to running Clang's
> output IR through `opt -O3` then `llc -O3`.
>

Thanks for pointing this out!

I built a local clang binary (to include a patch WIP) and ran `clang -v -x
ir -S -O3 <file.ll>`. It indeed worked equivalently to a three-step
operation (`opt` CLI with a patch | `opt` CLI to run CodeGenPrepare | `llc`)

>
> On Mon, Nov 15, 2021 at 1:43 PM Mingming Liu <mingmingl at google.com> wrote:
>
>> I used "llc -print-after-all -O3 <file.ll>" on this IR gives an assembly (
>> https://godbolt.org/z/K6cszrPPf), and codegenprepare indeed runs in
>> `llc` (from `print-after-all` output)
>>
>> The source of my confusion is:
>>
>>    1. Running the same IR by `opt -O3 -codegenprepare` gives a more
>>    optimized IR (https://godbolt.org/z/fdqTGsqG4)
>>    2. Piping the IR of step 1 (https://godbolt.org/z/544GMqaco) to `llc
>>    -O3` gives a better assembly (tail call generated).
>>
>> I'm missing something here..
>>
>> On Mon, Nov 15, 2021 at 1:00 PM Arthur Eubanks <aeubanks at google.com>
>> wrote:
>>
>>> `llc` should always run codegenprepare on IR before isel.
>>>
>>> On Mon, Nov 15, 2021 at 11:49 AM Mingming Liu <mingmingl at google.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Nov 15, 2021 at 10:34 AM Arthur Eubanks <aeubanks at google.com>
>>>> wrote:
>>>>
>>>>> `opt` is concerned about the optimization pipeline and `llc` is
>>>>> concerned about the codegen pipeline. codegenprepare is part of the codegen
>>>>> pipeline, not the optimization pipeline. We happen to be able to use `opt`
>>>>> to run codegenprepare on its own because of how legacy PM passes are
>>>>> structured and `llc` is not well suited to run individual IR passes.
>>>>>
>>>>
>>>> These all make sense to me.
>>>>
>>>> (The following idea side-tracks from the original topic, but just
>>>> brainstorming how to make the tools more friendly).
>>>>
>>>> If it (piping `opt` and `llc` misses `CodeGenPrepare` and causes
>>>> surprises) becomes a common question, `llc` tool might be enhanced by
>>>> emitting a warning/hint to CLI users that the IR probably needs
>>>> `CodeGenPrepare` pass (if input IR has metadata to record which middle-end
>>>> passes ran)
>>>>
>>>> This wouldn't change even if we used the NPM for the codegen pipeline.
>>>>>
>>>>
>>>> I get the point that CodeGenPrepare could be supported in `opt` (w/
>>>> NPM) since `opt` does IR to IR transformations.
>>>>
>>>>>
>>>>> On Fri, Nov 12, 2021 at 10:15 PM Mingming Liu via llvm-dev <
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> Thank you so much Arthur and Yuanfang! These pointers are very
>>>>>> educational.
>>>>>>
>>>>>> Now I realize there are two questions
>>>>>> 1) Use NPM for machine passes; this is the desired state RFC
>>>>>> <https://lists.llvm.org/pipermail/llvm-dev/2020-July/143309.html>
>>>>>>  and D85168 <https://reviews.llvm.org/D85168> tries to push forward.
>>>>>> 2) Whether CodeGenPrepare should be enabled by default (e.g., user of
>>>>>> opt CLI specifies an IR with sufficient target information, but doesn't
>>>>>> enable CodeGenPrepare explicitly).
>>>>>>
>>>>>> From
>>>>>> https://llvm.org/docs/NewPassManager.html#status-of-the-new-and-legacy-pass-managers,
>>>>>> the preferred option is to not run CodeGenPrepare in the default settings
>>>>>> (although users can still run it via specifying
>>>>>> *-passes=codegenprepare*).
>>>>>>
>>>>>> I could make sense of the pointers, and understood the rationales
>>>>>> better now.
>>>>>>
>>>>>> I'm curious if there were proposals to turn on CodeGenPrepare by
>>>>>> default (if IR has sufficient target information). (didn't find one with this
>>>>>> search query
>>>>>> <https://www.google.com/search?q=llvm+rfc+turning+on+codegenpreare+opt&newwindow=1&sxsrf=AOaemvIqK3A44HhoAdT538LwKCQ_tbhq1g%3A1636783711790&ei=X1aPYcPSL8rU-gSnoq-IDg&oq=llvm+rfc+turning+on+codegenpreare+opt&gs_lcp=Cgdnd3Mtd2l6EAMyBwgAEEcQsAMyBwgAEEcQsAMyBwgAEEcQsAMyBwgAEEcQsAMyBwgAEEcQsAMyBwgAEEcQsAMyBwgAEEcQsAMyBwgAEEcQsANKBAhBGABQAFgAYNYCaAFwAngAgAEAiAEAkgEAmAEAyAEIwAEB&sclient=gws-wiz&ved=0ahUKEwiD_tu91pT0AhVKqp4KHSfRC-EQ4dUDCA4&uact=5>
>>>>>> )
>>>>>> The good thing is that, when someone (e.g., like me when ramping up
>>>>>> on the llvm infra) pipes the *opt CLI* and *llc CLI *together, the
>>>>>> machine assembly is closer to the machine assembly of Clang (in cpp to
>>>>>> assembly mode).
>>>>>>
>>>>>> On Fri, Nov 12, 2021 at 2:17 PM <Yuanfang.Chen at sony.com> wrote:
>>>>>>
>>>>>>> Hi Mingming,
>>>>>>>
>>>>>>> About the status of using the new pass manager for the codegen
>>>>>>> pipeline, the RFC was here (
>>>>>>> http://lists.llvm.org/pipermail/llvm-dev/2020-July/143309.html) but
>>>>>>> there was no Bugzilla ticket for it, sorry! I've just created one
>>>>>>> https://bugs.llvm.org/show_bug.cgi?id=52493 with updates for anyone
>>>>>>> who might be interested. I haven't been able to follow up on it for a while
>>>>>>> but a few in-flight patches are still relevant and in good shape (check
>>>>>>> PR52493). I'll see if I could push them forward in the near future.
>>>>>>>
>>>>>>> About codegen-prepare, I don't have much to add other than Arthur's
>>>>>>> answer, except that D85168 would enable the use case, although it has some
>>>>>>> dependencies so it's not like that it could be landed soon.
>>>>>>>
>>>>>>> HTH,
>>>>>>> - Yuanfang
>>>>>>>
>>>>>>> ________________________________________
>>>>>>> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of
>>>>>>> Mingming Liu via llvm-dev <llvm-dev at lists.llvm.org>
>>>>>>> Sent: Friday, November 12, 2021 10:26 AM
>>>>>>> To: llvm-dev at lists.llvm.org
>>>>>>> Subject: [llvm-dev] status of CodeGen in new Pass Manager
>>>>>>>
>>>>>>> Hi,
>>>>>>>    This is a newbie question around CodeGen related passes and the
>>>>>>> current status in new Pass Manager.
>>>>>>>
>>>>>>>    From
>>>>>>> https://llvm.org/docs/NewPassManager.html#status-of-the-new-and-legacy-pass-managers
>>>>>>> <
>>>>>>> https://urldefense.com/v3/__https://llvm.org/docs/NewPassManager.html*status-of-the-new-and-legacy-pass-managers__;Iw!!JmoZiZGBv3RvKRSx!tI8u93htbfzW8OQkAVIdBlQTDHabCnLJtB2D5fD_OjBuK1ACPDpumEw6GK_dphuBDA$>,
>>>>>>> there are ongoing efforts to make the codegen pipeline work in the new Pass
>>>>>>> Manager (which is great!). Searching in the bug list (
>>>>>>> https://bugs.llvm.org/buglist.cgi?component=opt&list_id=226453&product=tools&query_format=advanced&resolution=---&short_desc=codegen&short_desc_type=allwordssubstr
>>>>>>> <
>>>>>>> https://urldefense.com/v3/__https://bugs.llvm.org/buglist.cgi?component=opt&list_id=226453&product=tools&query_format=advanced&resolution=---&short_desc=codegen&short_desc_type=allwordssubstr__;!!JmoZiZGBv3RvKRSx!tI8u93htbfzW8OQkAVIdBlQTDHabCnLJtB2D5fD_OjBuK1ACPDpumEw6GK-25d1S-w$>)
>>>>>>> gives no result.
>>>>>>>
>>>>>>>    I'm wondering if anyone has more information on the current
>>>>>>> status of CodeGen in the new Pass Manager (a tracking bug or other
>>>>>>> pointers)?
>>>>>>>
>>>>>>>    The context is that, I'm using opt CLI (by default new PM is
>>>>>>> used), and surprised that codegenprepare pass doesn't run, so dig down and
>>>>>>> having more questions :-)
>>>>>>>
>>>>>>>    Any related information will be appreciated!
>>>>>>>
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> Mingming
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Thanks,
>>>>>> Mingming
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> llvm-dev at lists.llvm.org
>>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Mingming
>>>>
>>>
>>
>> --
>> Thanks,
>> Mingming
>>
>

-- 
Thanks,
Mingming
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20211115/de878ec9/attachment.html>