[llvm-commits] [llvm] r157015 - in /llvm/trunk: ./ autoconf/ test/ test/ExecutionEngine/MCJIT/

[Danil Malyshev]

Hi Eli,

The idea of ​​using SUBTEST are interesting.
However, I want to voice some of the problems, because I do not know whether you discussed it with Daniel and if you have a solution for them.
1. Currently we can use the lit.local.cfg to choose the tests which should be executed for current platform, or should be skipped. If test should be skipped this test will be marked as UNSUPPORTED.
Inside the test, we can only choose whether we expect from the test that it will fails, or that is executed without error, the test will be executed in any case.
I think if some functionality should not work for current build, the tests for this functionality should not be executed at all and should be marked as UNSUPPORTED. This is a more valid option than "expected fail". Accordingly, it is necessary to move this functionality into the test to be able to use it for SUBTEST.
2. At present, the flag XFAIL checks the value of target_triple. That's not enough for the tests MCJIT, as, for example, among buildboots we have a cross-builds, which Windows host, and ARM target. In this case, the host platform is not supported by MCJIT, but the target platform is supported. So the MCJIT tests will be executed, will fails and will be marked as "unexpected fails".

Regards,
Danil


________________________________________
From: Bendersky, Eli [eli.bendersky at intel.com]
Sent: Saturday, May 19, 2012 9:56 PM
To: Jim Grosbach; Danil Malyshev; Daniel Dunbar (daniel.dunbar at gmail.com)
Cc: llvm-commits at cs.uiuc.edu for LLVM
Subject: RE: [llvm-commits] [llvm] r157015 - in /llvm/trunk: ./ autoconf/ test/ test/ExecutionEngine/MCJIT/

> Hi Danil,
>
> Do these really need to be copied into a new directory for the MCJIT? It
> seems we could run them for the MCJIT with a second RUN line in the
> original files instead. I semi-recall some discussion previously about doing
> something like that for some of the ELF tests. Eli, do you remember any of
> the details of that?

Jim,

We examined a number of ways to avoid duplicating the tests, but all of them stumbled on problems (i.e. the second RUN was infeasible because of limitations in lit for expecting failures for some RUNs). Eventually after some discussions with Daniel, I've started working on a more comprehensive change to lit for adding a SUBTEST  feature. An initial patch has already been reviewed by Daniel and I hope to have a fixed patch in the next few days. Once that's in lit, we'll be able to leverage this new functionality to cleanly run ExecutionEngine tests on both JIT and MCJIT with the same test file, without a need for duplication.

Eli




---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.