[llvm-dev] Significant code difference with a split call to opt

[Sébastien Michelland via llvm-dev]

Hi,

I reproduced the test on many individual files and got very variable 
results... it seems the computer's workload when running the test suite 
influenced the execution speed a lot more than standard deviation shows. 
I'll withdraw the performance claim until I can get consistent results 
(changed subject line), apologies for the confusion.

What I can still show easily is that the code generated by these two 
methods is different (which is already weird). For a simple example, 
grab a copy of bilateral_grid.bc:

 
<https://github.com/llvm/llvm-test-suite/blob/master/Bitcode/Benchmarks/Halide/bilateral_grid/bilateral_grid.bc>

Then you can generate my sequences with [opt -O3 -debug-pass=Arguments] 
and diff the outputs. Please see the attached script.

The differences seem to be mainly on variable indices (are they 
randomized?); on some test (namely jacobi-2d-imper) I have seen calling 
convention differences.

I'd like to optimize programs by greedily selecting optimizations, 
making a call to opt at each step. If I don't have equality between the 
two methods, I can't be sure that the sequence I'm building will make 
much sense.

Sébastien Michelland

On 6/14/19 4:49 PM, David Greene wrote:
> Do you have more information?  What were the exact command lines you
> used?  Do you have an example program that demonstrates the difference
> than you can share?
> 
>                        -David
> 
> Sébastien Michelland via llvm-dev <llvm-dev at lists.llvm.org> writes:
> 
>> Hello list,
>>
>> This is a follow-up from a question I asked last month. I'm evaluating
>> the performance of two pass sequences that resemble (but are not) -O3.
>>
>> With -O3, -debug-pass=Structure prints several independent blocks that
>> seem to represent several calls to opt. I focused on two of these
>> blocks, say S1 and S2, and compared the following optimization
>> methods:
>>
>> 1. Executing them separately, ie. opt -S1 | opt -S2
>> 2. Executing them in a single call, ie. opt -S1 -S2
>>
>> I built the test suite with each of these configurations, then
>> measured the performance of the compiled programs with perf, over 10
>> runs.
>>
>> I'm attaching a plot of the speedup of method 1 over method 2. The
>> intervals represent the standard deviation of the performance
>> measures.
>>
>> As you can see, programs compiled with method 1 are significantly
>> slower than their counterparts compiled with method 2. However, if
>> passes were applied in order using function composition, their
>> performance should be the same.
>>
>> I'd like to know if there is a way to recover this property in the
>> pass manager, or at least explain the difference. If needed, I can
>> provide scripts to reproduce the measurements.
>>
>> Thanks,
>> Sébastien Michelland
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build-bileteral-grid.sh
Type: application/x-shellscript
Size: 409 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190617/d1843710/attachment.bin>