<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: arial,helvetica,sans-serif; font-size: 10pt; color: #000000'><br><hr id="zwchr"><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"Alina Sbirlea via llvm-dev" <llvm-dev@lists.llvm.org><br><b>To: </b>"llvm-dev" <llvm-dev@lists.llvm.org><br><b>Sent: </b>Monday, February 29, 2016 1:16:54 PM<br><b>Subject: </b>Re: [llvm-dev] RFC: Add bitcode tests to test-suite<br><br><div dir="ltr">All,<div><br></div><div>To get the discussion going in a focused manner, here is an initial patch with a running test. The test is from the Halide suite and is checking the correctness of several simd operations.</div><div>(Notes: the patch is large due to the number of operations being tested; </div><div>I expect a lot of changes before actually landing it, this is simply to continue the discussion using a concrete example.)<br></div><div><a href="http://reviews.llvm.org/D17726" target="_blank">http://reviews.llvm.org/D17726</a><br></div><div><br></div><div>A few questions/todos to start the discussion:</div><div>1. What is a good location for these tests? They are in a separate Bitcode directory atm, but using the llvm_multisource. This may change to more closely model the approach for external tests (see next item). </div><div id="DWT15212">2. There is a single .cpp file testing all operations provided by individual bitcode files. I expect this to change. Instead of using llvm_multisource to have the same test run with specific arguments, each run testing a single operation.</div></div></blockquote><br>Thanks for continuing to work on this. Having drivers for these test bitcode files makes this significantly more appealing. This allows us to check that a crash fix was actually functionally correct, in addition to generally giving us more functional coverage.<br><br> -Hal<br><br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><div dir="ltr"><div></div><div>3. The building approach I took is to first link all bitcode files into a single one, then obtain the assembly for it, which cmake knows to take as an input source. </div><div>A separate discussion is on reading metadata (mcpu and mattr) in llc. I added a script to work around that for now.</div><div><br></div><div>Looking forward to your feedback!</div><div><br></div><div>Thanks,<br></div><div>Alina</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 19, 2016 at 6:50 AM, Kristof Beyls <span dir="ltr"><<a href="mailto:kristof.beyls@arm.com" target="_blank">kristof.beyls@arm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div><span class="">
<br>
<br>
<div>On 18/02/2016 19:12, Alina Sbirlea via
llvm-dev wrote:<br>
</div>
<blockquote>
<div dir="ltr"><br>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
I have more questions for Alina. What kind of tests do you
have:<br>
<br>
- "the compiler takes the bitcode and generates code
without crashing"<br>
- "the compiled test runs without crashing"<br>
- "the compiled test will produce an output that be
checked against a reference"<br>
- "the compiled test is meaningful as a benchmarks"<br>
</blockquote>
<div><br>
</div>
<div>We have all 4 kinds of tests in Halide. The bitcode
files for the first category is already available and I'm
working on building the ones for the next 3. We'd like to
include all incrementally.</div>
<div> </div>
<br>
</div>
</div>
</div>
</blockquote></span>
It seems to me that the first category ("the compiler takes the
bitcode and generates code without crashing") are tests that should
be added to the "make check-all" tests in the LLVM subproject,
rather than the test-suite subproject?<br>
Or if these tests currently don't crash the compiler anymore, the
bugs must have been fixed, and there should already be equivalent
tests?<br>
</div>
</blockquote></div><br></div>
<br>_______________________________________________<br>LLVM Developers mailing list<br>llvm-dev@lists.llvm.org<br>http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<br></blockquote><br><br><br>-- <br><div><span name="x"></span>Hal Finkel<br>Assistant Computational Scientist<br>Leadership Computing Facility<br>Argonne National Laboratory<span name="x"></span><br></div></div></body></html>