[llvm-dev] Statistics for Effectiveness of Passes on Reference Workloads

[Stefanos Baziotis via llvm-dev]

Hi Arthur,

`opt --print-changed(=quiet)`


I don't know how that argument "just works" now, I remember fighting it
like... two months ago :P For instance, opt 12.0.0:
https://godbolt.org/z/bd49oMqn5

You could add something to StandardInstrumentations to do something besides
> just printing the IR out when it changes.


Cool! I didn't know about that, thanks!

Best,
Stefanos

Στις Δευ, 24 Μαΐ 2021 στις 11:55 μ.μ., ο/η Arthur Eubanks <
aeubanks at google.com> έγραψε:

> For the new pass manager, `opt --print-changed(=quiet)` is very nice for
> filtering out passes that didn't do anything. You could add something to
> StandardInstrumentations to do something besides just printing the IR out
> when it changes.
>
> On Mon, May 24, 2021 at 1:52 PM Stefanos Baziotis via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi Min,
>>
>> Honestly, I don't think optimization remarks are good to check _whether_
>> a pass did any work. A far better option, to the best of my knowledge, is,
>> as I mentioned, the --print-after-all family [1]
>> Optimization remarks may help on the last question I posed and that is to
>> check _why_ a transformation was not applied. Although, if one reaches this
>> point, they will need to use way more
>> tools than remarks.
>>
>>  And even for the Passes that use it, my impression was that not every
>>> predicates are annotated with optimization remarks.
>>
>>
>> Definitely not and a lot of remarks are not that descriptive either.
>>
>> Best,
>> Stefanos
>>
>> [1] https://godbolt.org/z/58Ms7qW4s
>>
>> Στις Δευ, 24 Μαΐ 2021 στις 11:07 μ.μ., ο/η Min-Yih Hsu <minyihh at uci.edu>
>> έγραψε:
>>
>>> I think optimization remarks is a good framework for measuring whether a
>>> Pass does any work.
>>> But unfortunately it is not widely adopted by Passes outside loop
>>> transformations. And even for the Passes that use it, my impression was
>>> that not every predicates are annotated with optimization remarks.
>>>
>>> -Min
>>>
>>> On May 24, 2021, at 12:55 PM, Stefanos Baziotis via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>> Hi,
>>>
>>> Has anyone gathered statistics on reference workloads (*) for
>>> (transformation) passes
>>> that are enabled / run by default (e.g., -O3) but most of the
>>> time _don't_ do any effective transformation?
>>> Even better if we also have such statistics for passes that are _not_
>>> enabled by default
>>> (e.g., loop fusion, distribution, interchange, NewGVN).
>>>
>>> And yet even better if people have some idea / data for the reason for
>>> ineffectiveness.
>>> Bad heuristics / decision-making? Are some of these transformations
>>> useless most of the time?
>>> Or maybe they are useful but their implementation in LLVM is not
>>> powerful enough. Or maybe
>>> they incur a significant compile-time overhead.
>>>
>>> If not, it would also be helpful if anyone who has tried gathering
>>> similar statistics has any
>>> advice on how to approach it (my rough idea is initially
>>> use --print-changed / --print-after-all on these workloads for -O3 and
>>> then try to
>>> slide in passes that are not enabled by default; although that's harder
>>> to do it right).
>>>
>>> @Hideto: In the last LLVM meeting you gave a related talk [1]. Do you
>>> maybe have
>>> the full statistics and / or ways to reproduce them?
>>>
>>> Best,
>>> Stefanos
>>>
>>> (*)  SPEC, Polybench, Cryptographic libraries, Genome alignment, Image
>>> Processing, Graph Processing, Web browsers, databases like sqlite etc.
>>>
>>> [1] https://www.youtube.com/watch?v=QvF68tOt_w8
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210525/4452b985/attachment.html>