<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>You may also be interested in the following resources on
compilers correctness (articles, software, and talks -- from the
general topics to the ones specifically focused on testing,
validation, and verification):<br>
</p>
<p><a class="moz-txt-link-freetext" href="https://github.com/MattPD/cpplinks/blob/master/compilers.correctness.md">https://github.com/MattPD/cpplinks/blob/master/compilers.correctness.md</a></p>
<p>Best regards,</p>
<p>Matt P. Dziubinski<br>
</p>
<p>On 12/7/2018 20:19, Philip Reames via llvm-dev wrote:</p>
<blockquote type="cite"
cite="mid:7a207dec-4771-93ba-315d-e95988891068@philipreames.com">
<p>There's also a couple other active fuzzing efforts running
concurrently:</p>
<ul>
<li>OSSFuzz is continuously running a set of codegen and
individual pass fuzzers. A bit more information can be found
here:
<a class="moz-txt-link-freetext"
href="http://llvm.org/docs/FuzzingLLVM.html#continuously-running-and-finding-bugs"
moz-do-not-send="true">http://llvm.org/docs/FuzzingLLVM.html#continuously-running-and-finding-bugs</a></li>
<li>We (Azul Systems) have a continuously running Java fuzzer
which exercises LLVM through our Falcon compiler, and
regularly finds regressions and occasional deep long lurking
issues. We don't have a public bug tracker for this, but a
sizable portion of our upstream bug fixing activity is driven
by the output of this tool.</li>
</ul>
<p>Philip<br>
</p>
<div class="moz-cite-prefix">On 12/7/18 11:04 AM, Zhendong Su via
llvm-dev wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CALUzsy96TTRC2yLzSZs3p=aBjnazTM7Yr3Z2xWn8DF-P8_wNrw@mail.gmail.com">
<div dir="ltr">
<div dir="ltr">Thanks, Vedant. Yes, we have done a lot of
testing of Clang/LLVM (and GCC) in the past several years
(more details at <a
href="https://people.inf.ethz.ch/suz/emi/index.html"
moz-do-not-send="true">https://people.inf.ethz.ch/suz/emi/index.html</a>):
<div><br>
</div>
<div><span>[GCC/LLVM bugs: </span><b>1,602</b><span> (total)
/ </span><b>1,007</b><span> (fixed)]</span><br>
<span>[Reports: GCC (</span><a
href="http://tinyurl.com/ojzdt78" moz-do-not-send="true">link1</a><span>, </span><a
href="http://tinyurl.com/oxlkmjc" moz-do-not-send="true">link2</a><span>, </span><a
href="http://tinyurl.com/jd3jnl3" moz-do-not-send="true">link3</a><span>, </span><a
href="http://tinyurl.com/zs2826l" moz-do-not-send="true">link4</a><span>, </span><a
href="https://tinyurl.com/yb7okzqd"
moz-do-not-send="true">link5</a><span>), LLVM (</span><a
href="http://tinyurl.com/qj8x5ry" moz-do-not-send="true">link1</a><span>, </span><a
href="http://tinyurl.com/nc9celr" moz-do-not-send="true">link2</a><span>, </span><a
href="http://tinyurl.com/qy58smb" moz-do-not-send="true">link3</a><span>, </span><a
href="http://tinyurl.com/z7y8rwo" moz-do-not-send="true">link4</a><span>, </span><a
href="https://tinyurl.com/y7wdf5b4"
moz-do-not-send="true">link5</a><span>)]</span><br>
</div>
<div><br>
</div>
<div>
<div>Best,</div>
<div>Zhendong</div>
<div><br>
</div>
<div>--------</div>
<div>Zhendong Su<br>
</div>
<div>Professor, Department of Computer Science, ETH Zurich<br>
Office: CAB E 64.1, Tel: <span>+41 44 633 77 72</span></div>
<div><span><a href="https://people.inf.ethz.ch/suz/"
target="_blank" moz-do-not-send="true">https://people.inf.ethz.ch/suz/</a></span><br>
</div>
<div>--------</div>
</div>
<div><br>
</div>
<div>
<div class="gmail_quote">
<div dir="ltr">On Fri, Dec 7, 2018 at 7:42 PM Vedant
Kumar <<a href="mailto:vsk@apple.com"
moz-do-not-send="true">vsk@apple.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote">
<div>Hello,
<div><br>
</div>
<div>+ Zhendong, who’s done a lot of work on
automated testing of llvm.<br>
<div><br>
<blockquote type="cite">
<div>On Dec 6, 2018, at 10:15 PM, Radu Ometita
via llvm-dev <<a
href="mailto:llvm-dev@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-dev@lists.llvm.org</a>>
wrote:</div>
<br
class="gmail-m_6701961534287802269Apple-interchange-newline">
<div>
<div>
<div dir="auto">Hello everyone!<br>
<br>
We are working on writing a paper about
testing the reliability of C compilers
by using Csmith (a random C99 program
generator).<br>
<br>
A previous testing effort, using Csmith,
found 202 LLVM bugs, which represented
2% of all reported bugs at that time
(PDF: <a
href="https://www.flux.utah.edu/download?uid=114"
target="_blank" moz-do-not-send="true">https://www.flux.utah.edu/download?uid=114</a>):
. However, after this paper was
published we are unaware of any further
testing using Csmith, and we would like
to ask you if you are aware of any such
efforts or further results.<br>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Just speaking for myself here, I use Csmith
as part of my pre-commit testing.</div>
<div><br>
</div>
<div>vedant</div>
<br>
<blockquote type="cite">
<div>
<div>
<div dir="auto"><br>
Best regards,<br>
Radu Ometita,<br>
Functional compilers engineer @IOHK<br>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</blockquote>
<br>
</body>
</html>