[LLVMdev] qualitative comparison of correctness of llvm and gcc

[Kenneth Hoste]

Hello,

On Jul 20, 2008, at 23:19 , John Regehr wrote:

> These numbers come from throwing 100,000 random C programs at the
> various compilers.  For each compiler, we compiled each test case at
> -O0, -O1, -O2, -Os, and -O3 and compared the results of these
> compilations.  Random programs that crashed a compiler, and  
> executables
> that failed to terminate in a timely fashion, did not count towards  
> the
> 100,000.

Maybe I missed it in the follow-up discussion (I had to work through  
over 500 mails after getting back from holiday, so I wasn't always as  
thorough as I should), but I was wondering which technique/tool you  
used to generate these C programs. Maybe I'm being naive, but it  
doesn't seem like an easy task to generate non-trivial C programs to  
feed into a compiler...

Also, where would I be able to find more details on this experiment?  
That is, is there any paper or technical report on this? Or something  
else with more details on the setup of this?

greetings,

Kenneth Hoste
Phd student @ Ghent University, Belgium

-- 

Computer Science is no more about computers than astronomy is about  
telescopes. (E. W. Dijkstra)

Kenneth Hoste
ELIS - Ghent University
email: kenneth.hoste at elis.ugent.be
blog: http://www.elis.ugent.be/~kehoste/blog
website: http://www.elis.ugent.be/~kehoste