[llvm-dev] RFC: Add bitcode tests to test-suite

[Hal Finkel via llvm-dev]

----- Original Message -----

> From: "Alina Sbirlea via llvm-dev" <llvm-dev at lists.llvm.org>
> To: "llvm-dev" <llvm-dev at lists.llvm.org>
> Sent: Monday, February 29, 2016 1:16:54 PM
> Subject: Re: [llvm-dev] RFC: Add bitcode tests to test-suite

> All,

> To get the discussion going in a focused manner, here is an initial
> patch with a running test. The test is from the Halide suite and is
> checking the correctness of several simd operations.
> (Notes: the patch is large due to the number of operations being
> tested;
> I expect a lot of changes before actually landing it, this is simply
> to continue the discussion using a concrete example.)

> http://reviews.llvm.org/D17726

> A few questions/todos to start the discussion:
> 1. What is a good location for these tests? They are in a separate
> Bitcode directory atm, but using the llvm_multisource. This may
> change to more closely model the approach for external tests (see
> next item).
> 2. There is a single .cpp file testing all operations provided by
> individual bitcode files. I expect this to change. Instead of using
> llvm_multisource to have the same test run with specific arguments,
> each run testing a single operation.
Thanks for continuing to work on this. Having drivers for these test bitcode files makes this significantly more appealing. This allows us to check that a crash fix was actually functionally correct, in addition to generally giving us more functional coverage. 

-Hal 

> 3. The building approach I took is to first link all bitcode files
> into a single one, then obtain the assembly for it, which cmake
> knows to take as an input source.
> A separate discussion is on reading metadata (mcpu and mattr) in llc.
> I added a script to work around that for now.

> Looking forward to your feedback!

> Thanks,

> Alina

> On Fri, Feb 19, 2016 at 6:50 AM, Kristof Beyls <
> kristof.beyls at arm.com > wrote:

> > On 18/02/2016 19:12, Alina Sbirlea via llvm-dev wrote:
> 

> > > > I have more questions for Alina. What kind of tests do you
> > > > have:
> > > 
> > 
> 

> > > > - "the compiler takes the bitcode and generates code without
> > > > crashing"
> > > 
> > 
> 
> > > > - "the compiled test runs without crashing"
> > > 
> > 
> 
> > > > - "the compiled test will produce an output that be checked
> > > > against
> > > > a
> > > > reference"
> > > 
> > 
> 
> > > > - "the compiled test is meaningful as a benchmarks"
> > > 
> > 
> 

> > > We have all 4 kinds of tests in Halide. The bitcode files for the
> > > first category is already available and I'm working on building
> > > the
> > > ones for the next 3. We'd like to include all incrementally.
> > 
> 

> > It seems to me that the first category ("the compiler takes the
> > bitcode and generates code without crashing") are tests that should
> > be added to the "make check-all" tests in the LLVM subproject,
> > rather than the test-suite subproject?
> 
> > Or if these tests currently don't crash the compiler anymore, the
> > bugs must have been fixed, and there should already be equivalent
> > tests?
> 

> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 

Hal Finkel 
Assistant Computational Scientist 
Leadership Computing Facility 
Argonne National Laboratory 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160301/b55a1bb9/attachment.html>