[llvm-dev] Testing compiler reliability using Csmith
Philip Reames via llvm-dev
llvm-dev at lists.llvm.org
Fri Dec 7 11:19:47 PST 2018
There's also a couple other active fuzzing efforts running concurrently:
* OSSFuzz is continuously running a set of codegen and individual pass
fuzzers. A bit more information can be found here:
* We (Azul Systems) have a continuously running Java fuzzer which
exercises LLVM through our Falcon compiler, and regularly finds
regressions and occasional deep long lurking issues. We don't have
a public bug tracker for this, but a sizable portion of our upstream
bug fixing activity is driven by the output of this tool.
On 12/7/18 11:04 AM, Zhendong Su via llvm-dev wrote:
> Thanks, Vedant. Yes, we have done a lot of testing of Clang/LLVM (and
> GCC) in the past several years (more details at
> [GCC/LLVM bugs: *1,602* (total) / *1,007* (fixed)]
> [Reports: GCC (link1 <http://tinyurl.com/ojzdt78>, link2
> <http://tinyurl.com/oxlkmjc>, link3 <http://tinyurl.com/jd3jnl3>,
> link4 <http://tinyurl.com/zs2826l>, link5
> <https://tinyurl.com/yb7okzqd>), LLVM (link1
> <http://tinyurl.com/qj8x5ry>, link2 <http://tinyurl.com/nc9celr>,
> link3 <http://tinyurl.com/qy58smb>, link4
> <http://tinyurl.com/z7y8rwo>, link5 <https://tinyurl.com/y7wdf5b4>)]
> Zhendong Su
> Professor, Department of Computer Science, ETH Zurich
> Office: CAB E 64.1, Tel: +41 44 633 77 72
> On Fri, Dec 7, 2018 at 7:42 PM Vedant Kumar <vsk at apple.com
> <mailto:vsk at apple.com>> wrote:
> + Zhendong, who’s done a lot of work on automated testing of llvm.
>> On Dec 6, 2018, at 10:15 PM, Radu Ometita via llvm-dev
>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> Hello everyone!
>> We are working on writing a paper about testing the reliability
>> of C compilers by using Csmith (a random C99 program generator).
>> A previous testing effort, using Csmith, found 202 LLVM bugs,
>> which represented 2% of all reported bugs at that time (PDF:
>> https://www.flux.utah.edu/download?uid=114): . However, after
>> this paper was published we are unaware of any further testing
>> using Csmith, and we would like to ask you if you are aware of
>> any such efforts or further results.
> Just speaking for myself here, I use Csmith as part of my
> pre-commit testing.
>> Best regards,
>> Radu Ometita,
>> Functional compilers engineer @IOHK
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the llvm-dev