[llvm-dev] Statistics for Effectiveness of Passes on Reference Workloads

[Arthur Eubanks via llvm-dev]

For the new pass manager, `opt --print-changed(=quiet)` is very nice for
filtering out passes that didn't do anything. You could add something to
StandardInstrumentations to do something besides just printing the IR out
when it changes.

On Mon, May 24, 2021 at 1:52 PM Stefanos Baziotis via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi Min,
>
> Honestly, I don't think optimization remarks are good to check _whether_ a
> pass did any work. A far better option, to the best of my knowledge, is, as
> I mentioned, the --print-after-all family [1]
> Optimization remarks may help on the last question I posed and that is to
> check _why_ a transformation was not applied. Although, if one reaches this
> point, they will need to use way more
> tools than remarks.
>
>  And even for the Passes that use it, my impression was that not every
>> predicates are annotated with optimization remarks.
>
>
> Definitely not and a lot of remarks are not that descriptive either.
>
> Best,
> Stefanos
>
> [1] https://godbolt.org/z/58Ms7qW4s
>
> Στις Δευ, 24 Μαΐ 2021 στις 11:07 μ.μ., ο/η Min-Yih Hsu <minyihh at uci.edu>
> έγραψε:
>
>> I think optimization remarks is a good framework for measuring whether a
>> Pass does any work.
>> But unfortunately it is not widely adopted by Passes outside loop
>> transformations. And even for the Passes that use it, my impression was
>> that not every predicates are annotated with optimization remarks.
>>
>> -Min
>>
>> On May 24, 2021, at 12:55 PM, Stefanos Baziotis via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> Hi,
>>
>> Has anyone gathered statistics on reference workloads (*) for
>> (transformation) passes
>> that are enabled / run by default (e.g., -O3) but most of the
>> time _don't_ do any effective transformation?
>> Even better if we also have such statistics for passes that are _not_
>> enabled by default
>> (e.g., loop fusion, distribution, interchange, NewGVN).
>>
>> And yet even better if people have some idea / data for the reason for
>> ineffectiveness.
>> Bad heuristics / decision-making? Are some of these transformations
>> useless most of the time?
>> Or maybe they are useful but their implementation in LLVM is not powerful
>> enough. Or maybe
>> they incur a significant compile-time overhead.
>>
>> If not, it would also be helpful if anyone who has tried gathering
>> similar statistics has any
>> advice on how to approach it (my rough idea is initially
>> use --print-changed / --print-after-all on these workloads for -O3 and
>> then try to
>> slide in passes that are not enabled by default; although that's harder
>> to do it right).
>>
>> @Hideto: In the last LLVM meeting you gave a related talk [1]. Do you
>> maybe have
>> the full statistics and / or ways to reproduce them?
>>
>> Best,
>> Stefanos
>>
>> (*)  SPEC, Polybench, Cryptographic libraries, Genome alignment, Image
>> Processing, Graph Processing, Web browsers, databases like sqlite etc.
>>
>> [1] https://www.youtube.com/watch?v=QvF68tOt_w8
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210524/17680e4a/attachment.html>