[Openmp-dev] Switching testsuite to llvm-lit

[Sunita Chandrasekaran]

OK, thanks Jonathan.
Yes, right now the tests are checked twice.
-Sunita

On Thu, Jul 16, 2015 at 8:23 PM, Peyton, Jonathan L <
jonathan.l.peyton at intel.com> wrote:

>  From what I can tell, the current testsuite is triggered through
> Makefile targets to call runtest.pl which generates, then compiles, then
> runs the various tests, then at some point each test is compiled into the
> LLVM IR and RUN: and CHECK: lines are prepended to that generated file
> (.ll).  These are then run through lit.  So the tests get checked twice,
> once by the Perl system and once by lit.  What I want is for there to be
> plain ole C files that are the tests.  These C files should have the RUN:
> and CHECK: lines with no Perl required to generate the test files.  I’m
> currently making a concerted effort to have the entire build system
> Perl-free.  Another thing I’m confused about is the lit.site.cfg.in file
> which has to be configured to be used.  I don’t see any place where this is
> configured in the current system.  I may be missing it though.
>
>
>
> So, in short, I’m trying to get rid of the “middle-man” Perl step which
> generates, compiles, and runs the test files, by just having the test files
> be standalone C files which lit can handle.  Also, these changes would
> enable different generators to test OpenMP, for example Ninja build
> systems.  The tests themselves are great, and I truly appreciate their
> contribution, but they need to be better integrated into the current
> standalone CMake build system and the LLVM build system as a whole.
>
>
>
> -- Johnny
>
>
>
> *From:* sunisg123 at gmail.com [mailto:sunisg123 at gmail.com] *On Behalf Of *Sunita
> Chandrasekaran
> *Sent:* Thursday, July 16, 2015 6:44 PM
> *To:* Peyton, Jonathan L
> *Cc:* openmp-dev at dcs-maillist2.engr.illinois.edu
> *Subject:* Re: [Openmp-dev] Switching testsuite to llvm-lit
>
>
>
> Hi Jonathan
>
> Going through your steps, I am thinking this is close to what we (UH) had
> done and committed to the llvm trunk, especially creating those .cfg files
> and the RUN rules ?
>
> Sorry if I am mistaken.
>
>
>
> If what you are doing takes what we did and makes it more compatible to
> llvm-lit, I understand that. But I would hope that you don't have to start
> from scratch..
>
> Our testsuite was also using llvm-lit.
>
>
>
> -Sunita
>
>
>
>
>
> On Thu, Jul 16, 2015 at 4:01 PM, Peyton, Jonathan L <
> jonathan.l.peyton at intel.com> wrote:
>
>  Hello everyone,
>
>
>
> It has been requested, rightfully so, on more than one occasion that the
> testsuite be llvm-lit compatible.  This means calling llvm-lit directly
> inside the testsuite directory where it would pick up the configuration
> from lit.site.cfg and then run each test similar to other LLVM projects.
> Perl would neither exist nor be required as it is now for running the
> tests.  I was beginning to work on converting this and would like input,
> feedback on this process.
>
>
>
> What I’ve done so far includes:
>
> 1) Creating the tests --- Right now, the tests are stored as template
> files that must be translated by a perl script.  I’ve manually generated
> all the tests by running inside the testsuite/ directory
>
> $ template_parser_c.pl --test --noorphan c/$filename $filename
>
> This generates all the tests indentically to how runtest.pl does so.
>
> 2) I’ve then cleaned up the files, removing any reference to a logFile
> (llvm-lit will log for you) and cleaned up the typical generated code
> formatting problems so it looks like acceptable C code.
>
> 3) Then, I’ve created a basic lit.site.cfg.in file and a basic lit.cfg
> file (both based off of libcxx’s)
>
> 4) Lastly, I’ve been able to successfully run the tests using llvm-lit
> directly, but not yet from the build system itself as lit.site.cfg.in
> still needs work (I manually configure lit.site.cfg then run llvm-lit).
>
>
>
> Each test looks similar to the one below where the RUN: line says to
> compile the source file (%s) put it in a temp file (%t) , run the temp
> file, and compare the output of the temp file with the CHECK: line below.
> if Pass was printed, then the test passed, if not then the test failed.
>
>
>
> // RUN: %clang %openmp_flag %cflags %s -o %t && %t | FileCheck %s
>
> #include <stdio.h>
>
> #include "omp_testsuite.h"
>
> #include "omp_my_sleep.h"
>
>
>
> int test_omp_for_nowait()
>
> {
>
> [code to test it]
>
> }
>
>
>
> int main()
>
> {
>
>     int i;
>
>     int num_failed=0;
>
>
>
>     for(i = 0; i < REPETITIONS; i++) {
>
>         if(!test_omp_for_nowait()) {
>
>             num_failed++;
>
>         }
>
>     }
>
>
>
>     if(num_failed==0) {
>
>         printf("Pass\n");
>
>     } else {
>
>         printf("Failed %d out of %d times\n", num_failed, REPETITIONS);
>
>     }
>
>     // CHECK: Pass
>
>     return num_failed;
>
> }
>
>
>
> Any feedback or comments are welcome.  I was going to continue to work on
> it and get a first working iteration that could be run from a CMake build
> tree, then post a patch of the new files.
>
>
>
> -- Johnny
>
>
>
>
> _______________________________________________
> Openmp-dev mailing list
> Openmp-dev at dcs-maillist2.engr.illinois.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/openmp-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/openmp-dev/attachments/20150717/5f3d8a3e/attachment.html>