[LLVMdev] opt and bugs

[John Regehr]

A longstanding project on the LLVM "open projects" page is to run random 
C code through random LLVM passes.

We have found many LLVM bugs by doing the first of these, but not the 
second: we test only -O[0123s].

Slowly but surely, LLVM is evolving resistance to our random tester.  Of 
course this is good, but it means that we need to keep expanding our 
efforts if we want to continue to get solid return on investment for our 
bugfinding hours.

My first question is: When we start running random combinations of 
passes using opt, which ones should we leave out, because you folks 
don't want to hear that they're wrong?  For example, the other day I 
learned that ABCD is not yet ready for this sort of testing.  A while 
ago someone on the list said something similar about predsimplify 
(though it looks like that pass has disappeared from the tree?).

Also I'd appreciate some advice about probabilities for each pass.  For 
example, it seems like a lot of passes will have little to do if Mem2Reg 
doesn't run, so perhaps that pass should get >0.5 probability of 
running.  Does that make sense and do other examples come to mind?

Frankly I don't expect a zillion bugs to come out of this but it is 
super easy and you asked for it :).

Thanks,

John Regehr