[llvm-dev] Testing compiler reliability using Csmith

[Zhendong Su via llvm-dev]

Thanks, Vedant.  Yes, we have done a lot of testing of Clang/LLVM (and GCC)
in the past several years (more details at
https://people.inf.ethz.ch/suz/emi/index.html):

[GCC/LLVM bugs: *1,602* (total) / *1,007* (fixed)]
[Reports: GCC (link1 <http://tinyurl.com/ojzdt78>, link2
<http://tinyurl.com/oxlkmjc>, link3 <http://tinyurl.com/jd3jnl3>, link4
<http://tinyurl.com/zs2826l>, link5 <https://tinyurl.com/yb7okzqd>), LLVM (
link1 <http://tinyurl.com/qj8x5ry>, link2 <http://tinyurl.com/nc9celr>,
link3 <http://tinyurl.com/qy58smb>, link4 <http://tinyurl.com/z7y8rwo>,
link5 <https://tinyurl.com/y7wdf5b4>)]

Best,
Zhendong

--------
Zhendong Su
Professor, Department of Computer Science, ETH Zurich
Office: CAB E 64.1, Tel: +41 44 633 77 72
https://people.inf.ethz.ch/suz/
--------

On Fri, Dec 7, 2018 at 7:42 PM Vedant Kumar <vsk at apple.com> wrote:

> Hello,
>
> + Zhendong, who’s done a lot of work on automated testing of llvm.
>
> On Dec 6, 2018, at 10:15 PM, Radu Ometita via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hello everyone!
>
> We are working on writing a paper about testing the reliability of C
> compilers by using Csmith (a random C99 program generator).
>
> A previous testing effort, using Csmith, found 202 LLVM bugs, which
> represented 2% of all reported bugs at that time (PDF:
> https://www.flux.utah.edu/download?uid=114): . However, after this paper
> was published we are unaware of any further testing using Csmith, and we
> would like to ask you if you are aware of any such efforts or further
> results.
>
>
> Just speaking for myself here, I use Csmith as part of my pre-commit
> testing.
>
> vedant
>
>
> Best regards,
> Radu Ometita,
> Functional compilers engineer @IOHK
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181207/c71bda15/attachment.html>