[llvm-commits] [llvm] r157015 - in /llvm/trunk: ./ autoconf/ test/ test/ExecutionEngine/MCJIT/

[Bendersky, Eli]

> Hi Eli,
> 
> The idea of ​​using SUBTEST are interesting.
> However, I want to voice some of the problems, because I do not know
> whether you discussed it with Daniel and if you have a solution for them.
> 1. Currently we can use the lit.local.cfg to choose the tests which should be
> executed for current platform, or should be skipped. If test should be
> skipped this test will be marked as UNSUPPORTED.
> Inside the test, we can only choose whether we expect from the test that it
> will fails, or that is executed without error, the test will be executed in any
> case.
> I think if some functionality should not work for current build, the tests for
> this functionality should not be executed at all and should be marked as
> UNSUPPORTED. This is a more valid option than "expected fail". Accordingly,
> it is necessary to move this functionality into the test to be able to use it for
> SUBTEST.
> 2. At present, the flag XFAIL checks the value of target_triple. That's not
> enough for the tests MCJIT, as, for example, among buildboots we have a
> cross-builds, which Windows host, and ARM target. In this case, the host
> platform is not supported by MCJIT, but the target platform is supported. So
> the MCJIT tests will be executed, will fails and will be marked as "unexpected
> fails".
> 
> Regards,
> Danil

Hi Danil,

Thanks for the insights. Still, I think that expanding the infrastructure to support additional use cases is preferable to duplicating a ton of tests (I expect the ExecutionEngine test suite to grow considerably when MCJIT replaces JIT for more and more uses).

Regarding (1), I guess this is a philosophical discussion of XFAIL vs. UNSUPPORTED, which is appropriate when. In this case (of MCJIT), if we have tests that are currently failing but we generally expect them to be fixed and pass at some point in time, I don't see why XFAIL is inappropriate.

Regarding (2) - I must admit I didn't study the issue of cross builds, but here agai, the infrastructure (i.e. lit) can be enhanced to support this use case.

I'm attaching the latest patch I have that adds SUBTEST to lit (with some examples, so it should be fairly obvious how it works). If you could also take a look, and suggest how the special case(s) you mention can be fitted into this scheme, that would be great.

Eli







---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lit_subtest_4.patch
Type: application/octet-stream
Size: 16679 bytes
Desc: lit_subtest_4.patch
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20120522/1499c87a/attachment.obj>