[LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
Hal Finkel
hfinkel at anl.gov
Fri Jan 17 09:50:37 PST 2014
----- Original Message -----
> From: "Mohammad R Haghighat" <mohammad.r.haghighat at intel.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: llvmdev at cs.uiuc.edu
> Sent: Friday, January 17, 2014 11:47:01 AM
> Subject: RE: [LLVMdev] [icFuzz] Help needed with analyzing randomly generated tests that fail on clang 3.4 trunk
>
> Hi Hal,
>
> Just submitted 27 failing tests on clang version 3.5, trunk 199158.
> http://llvm.org/bugs/show_bug.cgi?id=16431
>
> I expect that these failures correspond to 2+ unique bugs.
Great, thanks! I'll work on reducing them in the next few days.
-Hal
>
> Cheers,
> -moh
>
> -----Original Message-----
> From: Hal Finkel [mailto:hfinkel at anl.gov]
> Sent: Thursday, January 09, 2014 6:01 AM
> To: Haghighat, Mohammad R
> Cc: llvmdev at cs.uiuc.edu
> Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly
> generated tests that fail on clang 3.4 trunk
>
> ----- Original Message -----
> > From: "Hal Finkel" <hfinkel at anl.gov>
> > To: "Mohammad R Haghighat" <mohammad.r.haghighat at intel.com>
> > Cc: llvmdev at cs.uiuc.edu
> > Sent: Sunday, July 28, 2013 11:01:14 PM
> > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing randomly
> > generated tests that fail on clang 3.4 trunk
> >
> > ----- Original Message -----
> > > Hal,
> > >
> > > Just posted a package containing 214 small tests showing bugs in
> > > the
> > > latest Clang (3.4 trunk 187225) on MacOS X when compiled at -O2.
> > > http://llvm.org/bugs/show_bug.cgi?id=16431
> > >
> > > These are new tests different from the previously posted ones,
> > > but
> > > their root causes could be the same as before or could actually
> > > be
> > > new bugs.
>
> Can you rerun icFuzz on the current trunk? At least most of the
> failures in the last batch were caused by an alias analysis bug that
> has now been fixed (thanks Arnold!).
>
> Thanks again,
> Hal
>
> >
> > Great, thanks! I'll go through them.
> >
> > -Hal
> >
> > >
> > > Cheers,
> > > -moh
> > >
> > > -----Original Message-----
> > > From: llvmdev-bounces at cs.uiuc.edu
> > > [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Haghighat,
> > > Mohammad R
> > > Sent: Friday, July 26, 2013 8:19 PM
> > > To: Hal Finkel
> > > Cc: llvmdev at cs.uiuc.edu
> > > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing
> > > randomly
> > > generated tests that fail on clang 3.4 trunk
> > >
> > > Hal,
> > >
> > > I ran the failing tests in the attachment to the bug 16431 on the
> > > latest clang trunk (version 3.4 trunk 187225).
> > > http://llvm.org/bugs/show_bug.cgi?id=16431
> > >
> > > The following tests still fail:
> > > Tests in diff: t10236 t12206 t2581 t6734
> > > t7788
> > > t7820 t8069 t9982
> > > All tests in InfLoopInClang: t19193 t22300 t25903 t27872
> > > t33143
> > > t8543
> > >
> > > Meanwhile, I'll launch a new run of icFuzz and will post the
> > > results
> > > later.
> > >
> > > -moh
> > >
> > > -----Original Message-----
> > > From: Hal Finkel [mailto:hfinkel at anl.gov]
> > > Sent: Friday, July 26, 2013 8:33 AM
> > > To: Haghighat, Mohammad R
> > > Cc: llvmdev at cs.uiuc.edu
> > > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing
> > > randomly
> > > generated tests that fail on clang 3.4 trunk
> > >
> > > ----- Original Message -----
> > > > ----- Original Message -----
> > > > > Great job, Hal!
> > > > >
> > > > > Sure. I'd be happy to run icFuzz and report the fails once
> > > > > these
> > > > > bugs are fixed and thereafter whenever people want new runs.
> > > > > Obviously, this can be automated, but the problem is that
> > > > > icFuzz
> > > > > is
> > > > > not currently open sourced.
> > > >
> > > > I would be happy to see this open sourced, but I think that we
> > > > can
> > > > work something out regardless.
> > > >
> > > > Also, once we get the current set of things resolved, I think
> > > > it
> > > > would
> > > > be useful to test running with:
> > > >
> > > > -- -O3, LTO (-O4 or -flto),
> > > > -- -fslp-vectorize, -fslp-vectorize-aggressive (which are
> > > > actually
> > > > separate optimizations)
> > > > -- -ffast-math (if you can do floating point with tolerances,
> > > > or
> > > > at
> > > > least -ffinite-math-only), -fno-math-errno (and there are
> > > > obviously a
> > > > whole bunch of non-default code-generation and target
> > > > options).
> > > >
> > > > Is it feasible to set up runs with different flags?
> > > >
> > > > > Once there's a bug in the compiler, there's really no limit
> > > > > in
> > > > > the
> > > > > number of failing tests that can be generated, so it's more
> > > > > productive to run the generator after the previously reported
> > > > > bugs
> > > > > are fixed.
> > > >
> > > > Agreed.
> > > >
> > > > >
> > > > > We've also seen cases where the results of "clang -O2" are
> > > > > different
> > > > > on Mac vs. Linux/Windows.
> > > >
> > > > I recall an issue related to default settings for FP, and
> > > > differences
> > > > with libm implementation. Are there non-floating-point cases?
> > > >
> > > > >
> > > > > Just let me know when you want a new run.
> > > >
> > > > Will do!
> > >
> > > Mohammad,
> > >
> > > Can you please re-run these now? I know that the original
> > > loop-vectorizer bugs causing the miscompiles have been fixed, and
> > > the others also seem to have been resolved as well.
> > >
> > > Thanks again,
> > > Hal
> > >
> > > >
> > > > -Hal
> > > >
> > > > >
> > > > > Cheers,
> > > > > -moh
> > > > >
> > > > > -----Original Message-----
> > > > > From: Hal Finkel [mailto:hfinkel at anl.gov]
> > > > > Sent: Wednesday, June 26, 2013 7:35 AM
> > > > > To: Haghighat, Mohammad R
> > > > > Cc: llvmdev at cs.uiuc.edu; Jim Grosbach
> > > > > Subject: Re: [LLVMdev] [icFuzz] Help needed with analyzing
> > > > > randomly
> > > > > generated tests that fail on clang 3.4 trunk
> > > > >
> > > > > ----- Original Message -----
> > > > > >
> > > > > > Hi Moh,
> > > > > >
> > > > > >
> > > > > > Thanks for this. I’m really glad to see the work you’re
> > > > > > doing
> > > > > > in
> > > > > > this area and believe it will be extremely helpful in
> > > > > > improving
> > > > > > the quality of the compiler.
> > > > > >
> > > > > >
> > > > > > -Jim
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Jun 24, 2013, at 4:10 PM, Haghighat, Mohammad R <
> > > > > > mohammad.r.haghighat at intel.com > wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I just submitted a bug report with a package containing 107
> > > > > > small
> > > > > > test cases that fail on the latest LLVM/clang 3.4 main
> > > > > > trunk
> > > > > > (184563). Included are test sources, compilation commands,
> > > > > > test
> > > > > > input files, and results at –O0 and –O2 when applicable.
> > > > > >
> > > > > > http://llvm.org/bugs/show_bug.cgi?id=16431
> > > > > >
> > > > > > These tests have been automatically generated by an
> > > > > > internal
> > > > > > tool
> > > > > > at Intel, the Intel Compiler fuzzer, icFuzz. The tests are
> > > > > > typically very small. For example, for the following simple
> > > > > > loop
> > > > > > (test
> > > > > > t5702)
> > > > > > on MacOS X, clang at –O2 generates a binary that crashes:
> > > > > >
> > > > > > // Test Loop Interchange
> > > > > > for (j = 2; j < 76; j++) {
> > > > > > for (jm = 1; jm < 30; jm++) {
> > > > > > h[j-1][jm-1] = j + 83;
> > > > > > }
> > > > > > }
> > > > > >
> > > > > > The tests are put in to two categories
> > > > > > - tests that have different runtime outputs when compiled
> > > > > > at
> > > > > > -O0
> > > > > > and
> > > > > > -O2 (this category also includes runtime crashes)
> > > > > > - tests that cause infinite loops in the Clang optimizer
> > > > > >
> > > > > > Many of these failing tests could be due to the same bug,
> > > > > > thus
> > > > > > a
> > > > > > much smaller number of root problems are expected.
> > > > > >
> > > > > > Any help with triaging these bugs would be highly
> > > > > > appreciated.
> > > > >
> > > > > I've gone through all of the miscompile cases, used bugpoint
> > > > > to
> > > > > reduce them, and opened individual PRs for several distinct
> > > > > bugs.
> > > > > So
> > > > > far we have: PR16455 (loop vectorizer), PR16457 (sccp),
> > > > > PR16460
> > > > > (instcombine). Thanks again for doing this! Do you plan on
> > > > > repeating
> > > > > this testing on a regular basis? Can it be automated?
> > > > >
> > > > > -Hal
> > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > -moh
> > > > > >
> > > > > > _______________________________________________
> > > > > > LLVM Developers mailing list
> > > > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > > > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > > > > >
> > > > > > _______________________________________________
> > > > > > LLVM Developers mailing list
> > > > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > > > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > > > > >
> > > > >
> > > > > --
> > > > > Hal Finkel
> > > > > Assistant Computational Scientist
> > > > > Leadership Computing Facility
> > > > > Argonne National Laboratory
> > > > >
> > > >
> > > > --
> > > > Hal Finkel
> > > > Assistant Computational Scientist
> > > > Leadership Computing Facility
> > > > Argonne National Laboratory
> > > >
> > > > _______________________________________________
> > > > LLVM Developers mailing list
> > > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > > >
> > >
> > > --
> > > Hal Finkel
> > > Assistant Computational Scientist
> > > Leadership Computing Facility
> > > Argonne National Laboratory
> > >
> > > _______________________________________________
> > > LLVM Developers mailing list
> > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > >
> >
> > --
> > Hal Finkel
> > Assistant Computational Scientist
> > Leadership Computing Facility
> > Argonne National Laboratory
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory
More information about the llvm-dev
mailing list