[LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk

[Karen Shaeffer]

On Mon, Jun 24, 2013 at 11:20:09PM +0000, Haghighat, Mohammad R wrote:
> The tests by design are syntactically correct, semantically correct, and have deterministic output.
> 
> -moh

Hi,
I wanted to believe you had a randomized code generator that could cover the entire
valid input space. (smiles ;) But the test cited was not generated by a randomized
code generator. It is way too structured. Even so, it could have been generated by a
partially randomized code generator that uses valid code idioms with some aspects
randomized. Interesting to think about it.

I wrote a fully randomized Java bytecode sequence generator once. It covered the entire
valid input space. I used it to test a Java hardware accelerator chip that had just been
taped out. It found quite a few bugs, where 4 of them were fatal to the chip.

Writing such a tool for a general purpose compiler would be quite a bit more
difficult, but it is doable. Imagine releasing compilers that are bug free,
at least for a given language and a particular platform. It's doable, and I wonder
why corporations don't invest in it.

enjoy,
Karen
-- 
Karen Shaeffer
Neuralscape, Mountain View, CA 94040