[LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
Haghighat, Mohammad R
mohammad.r.haghighat at intel.com
Fri Jan 17 09:47:01 PST 2014
Hi Hal,
Just submitted 27 failing tests on clang version 3.5, trunk 199158.
http://llvm.org/bugs/show_bug.cgi?id=16431
I expect that these failures correspond to 2+ unique bugs.
Cheers,
-moh
-----Original Message-----
From: Hal Finkel [mailto:hfinkel at anl.gov]
Sent: Thursday, January 09, 2014 6:01 AM
To: Haghighat, Mohammad R
Cc: llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
----- Original Message -----
> From: "Hal Finkel" <hfinkel at anl.gov>
> To: "Mohammad R Haghighat" <mohammad.r.haghighat at intel.com>
> Cc: llvmdev at cs.uiuc.edu
> Sent: Sunday, July 28, 2013 11:01:14 PM
> Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
>
> ----- Original Message -----
> > Hal,
> >
> > Just posted a package containing 214 small tests showing bugs in
> > the
> > latest Clang (3.4 trunk 187225) on MacOS X when compiled at -O2.
> > http://llvm.org/bugs/show_bug.cgi?id=16431
> >
> > These are new tests different from the previously posted ones, but
> > their root causes could be the same as before or could actually be
> > new bugs.
Can you rerun icFuzz on the current trunk? At least most of the failures in the last batch were caused by an alias analysis bug that has now been fixed (thanks Arnold!).
Thanks again,
Hal
>
> Great, thanks! I'll go through them.
>
> -Hal
>
> >
> > Cheers,
> > -moh
> >
> > -----Original Message-----
> > From: llvmdev-bounces at cs.uiuc.edu
> > [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Haghighat,
> > Mohammad R
> > Sent: Friday, July 26, 2013 8:19 PM
> > To: Hal Finkel
> > Cc: llvmdev at cs.uiuc.edu
> > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly
> > generated tests that fail on clang 3.4 trunk
> >
> > Hal,
> >
> > I ran the failing tests in the attachment to the bug 16431 on the
> > latest clang trunk (version 3.4 trunk 187225).
> > http://llvm.org/bugs/show_bug.cgi?id=16431
> >
> > The following tests still fail:
> > Tests in diff: t10236 t12206 t2581 t6734 t7788
> > t7820 t8069 t9982
> > All tests in InfLoopInClang: t19193 t22300 t25903 t27872 t33143
> > t8543
> >
> > Meanwhile, I'll launch a new run of icFuzz and will post the
> > results
> > later.
> >
> > -moh
> >
> > -----Original Message-----
> > From: Hal Finkel [mailto:hfinkel at anl.gov]
> > Sent: Friday, July 26, 2013 8:33 AM
> > To: Haghighat, Mohammad R
> > Cc: llvmdev at cs.uiuc.edu
> > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly
> > generated tests that fail on clang 3.4 trunk
> >
> > ----- Original Message -----
> > > ----- Original Message -----
> > > > Great job, Hal!
> > > >
> > > > Sure. I'd be happy to run icFuzz and report the fails once
> > > > these
> > > > bugs are fixed and thereafter whenever people want new runs.
> > > > Obviously, this can be automated, but the problem is that
> > > > icFuzz
> > > > is
> > > > not currently open sourced.
> > >
> > > I would be happy to see this open sourced, but I think that we
> > > can
> > > work something out regardless.
> > >
> > > Also, once we get the current set of things resolved, I think it
> > > would
> > > be useful to test running with:
> > >
> > > -- -O3, LTO (-O4 or -flto),
> > > -- -fslp-vectorize, -fslp-vectorize-aggressive (which are
> > > actually
> > > separate optimizations)
> > > -- -ffast-math (if you can do floating point with tolerances, or
> > > at
> > > least -ffinite-math-only), -fno-math-errno (and there are
> > > obviously a
> > > whole bunch of non-default code-generation and target options).
> > >
> > > Is it feasible to set up runs with different flags?
> > >
> > > > Once there's a bug in the compiler, there's really no limit in
> > > > the
> > > > number of failing tests that can be generated, so it's more
> > > > productive to run the generator after the previously reported
> > > > bugs
> > > > are fixed.
> > >
> > > Agreed.
> > >
> > > >
> > > > We've also seen cases where the results of "clang -O2" are
> > > > different
> > > > on Mac vs. Linux/Windows.
> > >
> > > I recall an issue related to default settings for FP, and
> > > differences
> > > with libm implementation. Are there non-floating-point cases?
> > >
> > > >
> > > > Just let me know when you want a new run.
> > >
> > > Will do!
> >
> > Mohammad,
> >
> > Can you please re-run these now? I know that the original
> > loop-vectorizer bugs causing the miscompiles have been fixed, and
> > the others also seem to have been resolved as well.
> >
> > Thanks again,
> > Hal
> >
> > >
> > > -Hal
> > >
> > > >
> > > > Cheers,
> > > > -moh
> > > >
> > > > -----Original Message-----
> > > > From: Hal Finkel [mailto:hfinkel at anl.gov]
> > > > Sent: Wednesday, June 26, 2013 7:35 AM
> > > > To: Haghighat, Mohammad R
> > > > Cc: llvmdev at cs.uiuc.edu; Jim Grosbach
> > > > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing
> > > > randomly
> > > > generated tests that fail on clang 3.4 trunk
> > > >
> > > > ----- Original Message -----
> > > > >
> > > > > Hi Moh,
> > > > >
> > > > >
> > > > > Thanks for this. I’m really glad to see the work you’re doing
> > > > > in
> > > > > this area and believe it will be extremely helpful in
> > > > > improving
> > > > > the quality of the compiler.
> > > > >
> > > > >
> > > > > -Jim
> > > > >
> > > > >
> > > > >
> > > > > On Jun 24, 2013, at 4:10 PM, Haghighat, Mohammad R <
> > > > > mohammad.r.haghighat at intel.com > wrote:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > I just submitted a bug report with a package containing 107
> > > > > small
> > > > > test cases that fail on the latest LLVM/clang 3.4 main trunk
> > > > > (184563). Included are test sources, compilation commands,
> > > > > test
> > > > > input files, and results at –O0 and –O2 when applicable.
> > > > >
> > > > > http://llvm.org/bugs/show_bug.cgi?id=16431
> > > > >
> > > > > These tests have been automatically generated by an internal
> > > > > tool
> > > > > at Intel, the Intel Compiler fuzzer, icFuzz. The tests are
> > > > > typically very small. For example, for the following simple
> > > > > loop
> > > > > (test
> > > > > t5702)
> > > > > on MacOS X, clang at –O2 generates a binary that crashes:
> > > > >
> > > > > // Test Loop Interchange
> > > > > for (j = 2; j < 76; j++) {
> > > > > for (jm = 1; jm < 30; jm++) {
> > > > > h[j-1][jm-1] = j + 83;
> > > > > }
> > > > > }
> > > > >
> > > > > The tests are put in to two categories
> > > > > - tests that have different runtime outputs when compiled at
> > > > > -O0
> > > > > and
> > > > > -O2 (this category also includes runtime crashes)
> > > > > - tests that cause infinite loops in the Clang optimizer
> > > > >
> > > > > Many of these failing tests could be due to the same bug,
> > > > > thus
> > > > > a
> > > > > much smaller number of root problems are expected.
> > > > >
> > > > > Any help with triaging these bugs would be highly
> > > > > appreciated.
> > > >
> > > > I've gone through all of the miscompile cases, used bugpoint to
> > > > reduce them, and opened individual PRs for several distinct
> > > > bugs.
> > > > So
> > > > far we have: PR16455 (loop vectorizer), PR16457 (sccp), PR16460
> > > > (instcombine). Thanks again for doing this! Do you plan on
> > > > repeating
> > > > this testing on a regular basis? Can it be automated?
> > > >
> > > > -Hal
> > > >
> > > > >
> > > > > Thanks,
> > > > > -moh
> > > > >
> > > > > _______________________________________________
> > > > > LLVM Developers mailing list
> > > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > > > >
> > > > > _______________________________________________
> > > > > LLVM Developers mailing list
> > > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > > > >
> > > >
> > > > --
> > > > Hal Finkel
> > > > Assistant Computational Scientist
> > > > Leadership Computing Facility
> > > > Argonne National Laboratory
> > > >
> > >
> > > --
> > > Hal Finkel
> > > Assistant Computational Scientist
> > > Leadership Computing Facility
> > > Argonne National Laboratory
> > >
> > > _______________________________________________
> > > LLVM Developers mailing list
> > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > >
> >
> > --
> > Hal Finkel
> > Assistant Computational Scientist
> > Leadership Computing Facility
> > Argonne National Laboratory
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory
More information about the llvm-dev
mailing list