<div dir="ltr">OK, thanks Jonathan. <div>Yes, right now the tests are checked twice. </div><div>-Sunita<br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 16, 2015 at 8:23 PM, Peyton, Jonathan L <span dir="ltr"><<a href="mailto:jonathan.l.peyton@intel.com" target="_blank">jonathan.l.peyton@intel.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">From what I can tell, the current testsuite is triggered through Makefile targets to call <a href="http://runtest.pl" target="_blank">runtest.pl</a> which generates, then compiles, then runs the various tests,
then at some point each test is compiled into the LLVM IR and RUN: and CHECK: lines are prepended to that generated file (.ll). These are then run through lit. So the tests get checked twice, once by the Perl system and once by lit. What I want is for there
to be plain ole C files that are the tests. These C files should have the RUN: and CHECK: lines with no Perl required to generate the test files. I’m currently making a concerted effort to have the entire build system Perl-free. Another thing I’m confused
about is the <a href="http://lit.site.cfg.in" target="_blank">lit.site.cfg.in</a> file which has to be configured to be used. I don’t see any place where this is configured in the current system. I may be missing it though.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">So, in short, I’m trying to get rid of the “middle-man” Perl step which generates, compiles, and runs the test files, by just having the test files be standalone
C files which lit can handle. Also, these changes would enable different generators to test OpenMP, for example Ninja build systems. The tests themselves are great, and I truly appreciate their contribution, but they need to be better integrated into the
current standalone CMake build system and the LLVM build system as a whole.<u></u><u></u></span></p>
<p class="MsoNormal"><a name="14e9bd27fa9b5736_14e999c127876c9a__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></a></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">-- Johnny<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> <a href="mailto:sunisg123@gmail.com" target="_blank">sunisg123@gmail.com</a> [mailto:<a href="mailto:sunisg123@gmail.com" target="_blank">sunisg123@gmail.com</a>]
<b>On Behalf Of </b>Sunita Chandrasekaran<br>
<b>Sent:</b> Thursday, July 16, 2015 6:44 PM<br>
<b>To:</b> Peyton, Jonathan L<br>
<b>Cc:</b> <a href="mailto:openmp-dev@dcs-maillist2.engr.illinois.edu" target="_blank">openmp-dev@dcs-maillist2.engr.illinois.edu</a><br>
<b>Subject:</b> Re: [Openmp-dev] Switching testsuite to llvm-lit<u></u><u></u></span></p><div><div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hi Jonathan <u></u><u></u></p>
<div>
<p class="MsoNormal">Going through your steps, I am thinking this is close to what we (UH) had done and committed to the llvm trunk, especially creating those .cfg files and the RUN rules ? <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Sorry if I am mistaken. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">If what you are doing takes what we did and makes it more compatible to llvm-lit, I understand that. But I would hope that you don't have to start from scratch..<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Our testsuite was also using llvm-lit. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">-Sunita<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Thu, Jul 16, 2015 at 4:01 PM, Peyton, Jonathan L <<a href="mailto:jonathan.l.peyton@intel.com" target="_blank">jonathan.l.peyton@intel.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal">Hello everyone,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">It has been requested, rightfully so, on more than one occasion that the testsuite be llvm-lit compatible. This means calling llvm-lit directly inside the testsuite directory where
it would pick up the configuration from lit.site.cfg and then run each test similar to other LLVM projects. Perl would neither exist nor be required as it is now for running the tests. I was beginning to work on converting this and would like input, feedback
on this process.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">What I’ve done so far includes:<u></u><u></u></p>
<p class="MsoNormal">1) Creating the tests --- Right now, the tests are stored as template files that must be translated by a perl script. I’ve manually generated all the tests by running inside the
testsuite/ directory<u></u><u></u></p>
<p class="MsoNormal">$
<a href="http://template_parser_c.pl" target="_blank">template_parser_c.pl</a> --test --noorphan c/$filename $filename<u></u><u></u></p>
<p class="MsoNormal">This generates all the tests indentically to how
<a href="http://runtest.pl" target="_blank">runtest.pl</a> does so.<u></u><u></u></p>
<p class="MsoNormal">2) I’ve then cleaned up the files, removing any reference to a logFile (llvm-lit will log for you) and cleaned up the typical generated code formatting problems so it looks like
acceptable C code.<u></u><u></u></p>
<p class="MsoNormal">3) Then, I’ve created a basic
<a href="http://lit.site.cfg.in" target="_blank">lit.site.cfg.in</a> file and a basic lit.cfg file (both based off of libcxx’s)<u></u><u></u></p>
<p class="MsoNormal">4) Lastly, I’ve been able to successfully run the tests using llvm-lit directly, but not yet from the build system itself as
<a href="http://lit.site.cfg.in" target="_blank">lit.site.cfg.in</a> still needs work (I manually configure lit.site.cfg then run llvm-lit).<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Each test looks similar to the one below where the RUN: line says to compile the source file (%s) put it in a temp file (%t) , run the temp file, and compare the output of the temp
file with the CHECK: line below. if Pass was printed, then the test passed, if not then the test failed.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">// RUN: %clang %openmp_flag %cflags %s -o %t && %t | FileCheck %s</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">#include <stdio.h></span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">#include "omp_testsuite.h"</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">#include "omp_my_sleep.h"</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> </span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">int test_omp_for_nowait()</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">{</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">[code to test it]</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">}</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> </span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">int main()</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">{</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> int i;</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> int num_failed=0;</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> </span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> for(i = 0; i < REPETITIONS; i++) {</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> if(!test_omp_for_nowait()) {</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> num_failed++;</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> }</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> }</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> </span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> if(num_failed==0) {</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> printf("Pass\n");</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> } else {</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> printf("Failed %d out of %d times\n", num_failed, REPETITIONS);</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> }</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> // CHECK: Pass</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas"> return num_failed;</span><u></u><u></u></p>
<p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:Consolas">}</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Any feedback or comments are welcome. I was going to continue to work on it and get a first working iteration that could be run from a CMake build tree, then post a patch of the
new files.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">-- Johnny<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Openmp-dev mailing list<br>
<a href="mailto:Openmp-dev@dcs-maillist2.engr.illinois.edu" target="_blank">Openmp-dev@dcs-maillist2.engr.illinois.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/openmp-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/openmp-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div></div></div>
</div>
</blockquote></div><br></div></div></div>