Platform-specific RUN lines in lit tests?

[Alp Toker]

Hi,

It would seem expedient to use the standard clang driver on all 
platforms instead of clang-cl.exe on Windows.

They both ultimately invoke the same -cc1 options with full MSVC 
compatibility, while clang-cl.exe primarily exists to ease IDE 
integration so I don't get the usage here.

As for custom RUN lines, we tried that some time back (and I think 
posted a patch to llvmdev). It kind of goes against lit's conceptual 
model. If there is some way you can get these cases covered manually 
without too much pain that might be preferable.

Alp.


On 14/05/2014 21:20, Alexey Samsonov wrote:
> +Daniel, Alp
>
> On Wed, May 14, 2014 at 10:00 AM, David Blaikie <dblaikie at gmail.com 
> <mailto:dblaikie at gmail.com>> wrote:
>
>     On Wed, May 14, 2014 at 9:43 AM, Timur Iskhodzhanov
>     <timurrrr at google.com <mailto:timurrrr at google.com>> wrote:
>     > Hi,
>     >
>     > I'm currently enabling ASan lit tests on Windows.
>     >
>     > One problem I have is that some tests use multiple RUN lines,
>     some of which
>     > are cross-platform and others are platform- or compiler-specific.
>     > Is there any magic method to make some of the RUN lines
>     conditional for a
>     > specific OS?
>
>
> In fact, I'd vote for adding this feature to lit :) Having
>   RUN-Windows:
> would be nice. We can parse
>   RUN-<foo>:
> lines and use them in the test script only if <foo> is in the list of 
> available features.
> Otherwise we'll have to copy the tests around or do what David 
> suggests, which is fine,
> but makes the tests less readable and less self-contained.
>
>     >
>     > I think one of the goals here is to have as many tests that are
>     actually
>     > shared on all the platforms and thus I don't think it's a very
>     good idea to
>     > e.g. duplicate tests and have platform-specific RUN lines in them.
>     >
>     > Examples to consider are:
>     > [all located in projects/compiler/rt/test/asan/TestCases/]
>     >
>     > 1) sanity_check_pure_c.c has the following run lines:
>     > // RUN: %clang_asan -O2 %s -o %t
>     > // RUN: not %run %t 2>&1 | FileCheck %s
>     > // RUN: %clang_asan -O2 %s -pie -fPIE -o %t
>     > // RUN: not %run %t 2>&1 | FileCheck %s
>     >
>     > The first two lines are ok to run on any OS.
>     > The third run line uses "-pie -fPIE" which doesn't apply on Windows.
>     > Is there any way to disable the third and fourth RUN lines on
>     Windows only?
>     >
>     > 2) print-stack-trace.cc has the following run lines:
>     > // RUN: %clangxx_asan -O0 %s -o %t && %run %t 2>&1 | FileCheck %s
>     > // RUN: %clangxx_asan -O3 %s -o %t && %run %t 2>&1 | FileCheck %s
>     >
>     > The test expectations assume a particular stack trace being printed.
>     > However, on Windows -O3 enables tail call / inlining
>     optimizations that do
>     > not allow us to completely restore the stack trace from debug
>     information
>     > (yet?).
>
>
> Maybe, it's ok to XFAIL this test on Windows for now.
>
>     >
>     > 3) interface_test.cc has the following run lines:
>     > // RUN: %clang_asan %s -o %t && %run %t
>     > // RUN: %clang_asan -x c %s -o %t && %run %t
>     > // RUN: %clang %s -o %t && %run %t
>     > // RUN: %clang -x c %s -o %t && %run %t
>     >
>     > the first and third are ok, the second and the fourth assume
>     clang has "-x
>     > c" available, which is not true on Windows where we use the
>     clang-cl driver.
>     > The equivalent CL/clang-cl option is "/TC".
>     > Options are:
>     > - Copy the test to interface_test.c
>
>     You don't need to copy the whole test - you can just write another
>     file (interface_test_windows.test, or whatever) that, in its RUN line,
>     references interface_test.c (you might have to move interface_test.c
>     into an "Inputs" directory, then you would have interface.test and
>     interface_windows.test that both reference the .c file in Inputs) -
>     and one of those .test files would have the REQUIRES: blah and have
>     the RUN lines that aren't portable in it.
>
>     > - Use a macro for "-x c" / "/TC"  (sounds like an overkill for
>     just one
>     > test?)
>
>
> Yes, sounds like an overkill. Is it possible to support "-x" option in 
> clang-cl driver?
>
>
>     This seems like an annoying thing to deal with - is it particularly
>     useful to test the clang-cl driver rather than just using the normal
>     clang gcc-like driver?
>
>     > - Conditional RUN line
>     > ...
>     >
>     > 4) no_asan_gen_globals.c
>     > // RUN: %clang_asan %s -o %t.exe
>     > // RUN: nm %t.exe | FileCheck %s
>     > -> Could probably use llvm-something.exe instead.
>
>
> Yep, I'd check if llvm-nm can handle this.
>
>     >
>     > -----
>     >
>     > Given that there's just a handful of problematic lit tests, I
>     kinda think we
>     > can actually duplicate the tests a little, but I'd like to avoid
>     that if
>     > possible.
>     >
>     > Any tips?
>     >
>     > --
>     > Tim
>     >
>     > _______________________________________________
>     > llvm-commits mailing list
>     > llvm-commits at cs.uiuc.edu <mailto:llvm-commits at cs.uiuc.edu>
>     > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>     >
>
>
>
>
> -- 
> Alexey Samsonov, Mountain View, CA

-- 
http://www.nuanti.com
the browser experts