[LLVMdev] qualitative comparison of correctness of llvm and gcc

[Bill Wendling]

On Jul 20, 2008, at 2:19 PM, John Regehr wrote:

> Hi folks,
>
> We recently generated some data that seemed interesting enough to  
> share
> here.  This is a comparison between compilers that ignores the
> performance of the generated code and focuses only on compiler  
> correctness.
>
>                   volatile    checksum
>                     errors      errors
>
> avr-gcc-3.4         1.879%      0.378%
> avr-gcc-4.1         0.037%      0.256%
> avr-gcc-4.2         0.753%      0.225%
> gcc-3.4             1.222%      0.004%
> gcc-4.0             0.041%      0.026%
> gcc-4.1             0.196%      0.024%
> gcc-4.2             0.770%      0.004%
> gcc-4.3             0.727%      0.003%
> llvm-gcc-2.2       18.712%      0.110%
> llvm-gcc-pre-2.4    0.000%      0.006%
>
John,

These numbers are terrific! Thank you for compiling them and for the  
bugs you filed. From the list below, it looks like there are only 2  
outstanding bugs -- one in checksum and one non-terminating  
compilation. Very nice!

-bw