[LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk

[Haghighat, Mohammad R]

Great job, Hal! 

Sure. I'd be happy to run icFuzz and report the fails once these bugs are fixed and thereafter whenever people want new runs. Obviously, this can be automated, but the problem is that icFuzz is not currently open sourced. Once there's a bug in the compiler, there's really no limit in the number of failing tests that can be generated, so it's more productive to run the generator after the previously reported bugs are fixed.

We've also seen cases where the results of "clang -O2" are different on Mac vs. Linux/Windows.

Just let me know when you want a new run.

Cheers,
-moh

-----Original Message-----
From: Hal Finkel [mailto:hfinkel at anl.gov] 
Sent: Wednesday, June 26, 2013 7:35 AM
To: Haghighat, Mohammad R
Cc: llvmdev at cs.uiuc.edu; Jim Grosbach
Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk

----- Original Message -----
> 
> Hi Moh,
> 
> 
> Thanks for this. I’m really glad to see the work you’re doing in this
> area and believe it will be extremely helpful in improving the
> quality of the compiler.
> 
> 
> -Jim
> 
> 
> 
> On Jun 24, 2013, at 4:10 PM, Haghighat, Mohammad R <
> mohammad.r.haghighat at intel.com > wrote:
> 
> 
> 
> 
> 
> Hi,
> 
> I just submitted a bug report with a package containing 107 small
> test cases that fail on the latest LLVM/clang 3.4 main trunk
> (184563). Included are test sources, compilation commands, test
> input files, and results at –O0 and –O2 when applicable.
> 
> http://llvm.org/bugs/show_bug.cgi?id=16431
> 
> These tests have been automatically generated by an internal tool at
> Intel, the Intel Compiler fuzzer, icFuzz. The tests are typically
> very small. For example, for the following simple loop (test t5702)
> on MacOS X, clang at –O2 generates a binary that crashes:
> 
> // Test Loop Interchange
> for (j = 2; j < 76; j++) {
> for (jm = 1; jm < 30; jm++) {
> h[j-1][jm-1] = j + 83;
> }
> }
> 
> The tests are put in to two categories
> - tests that have different runtime outputs when compiled at -O0 and
> -O2 (this category also includes runtime crashes)
> - tests that cause infinite loops in the Clang optimizer
> 
> Many of these failing tests could be due to the same bug, thus a much
> smaller number of root problems are expected.
> 
> Any help with triaging these bugs would be highly appreciated.

I've gone through all of the miscompile cases, used bugpoint to reduce them, and opened individual PRs for several distinct bugs. So far we have: PR16455 (loop vectorizer), PR16457 (sccp), PR16460 (instcombine). Thanks again for doing this! Do you plan on repeating this testing on a regular basis? Can it be automated?

 -Hal

> 
> Thanks,
> -moh
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory