[llvm] [Offload] Run unit tests as a part of check-offload (PR #156675)

[Joseph Huber via llvm-commits]

jhuber6 wrote:

> With #157253 I was able to use a `runtimes` build for offload, but then I'm seeing a fair bunch of unittests fail with errors such as:
> 
> ```
> FAIL: Offload-Unit :: OffloadAPI/./device.unittests/2/4 (4 of 44)
> ******************** TEST 'Offload-Unit :: OffloadAPI/./device.unittests/2/4' FAILED ********************
> Script(shard):
> --
> GTEST_OUTPUT=json:/var/tmp/portage/llvm-runtimes/offload-22.0.0.9999/work/runtimes_build/offload/unittests/OffloadAPI/./device.unittests-Offload-Unit-209-2-4.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=4 GTEST_SHARD_INDEX=2 /var/tmp/portage/llvm-runtimes/offload-22.0.0.9999/work/runtimes_build/offload/unittests/OffloadAPI/./device.unittests
> --
> 
> Script:
> --
> /var/tmp/portage/llvm-runtimes/offload-22.0.0.9999/work/runtimes_build/offload/unittests/OffloadAPI/./device.unittests --gtest_filter=GoogleTestVerification.UninstantiatedParameterizedTestSuite<olGetDeviceInfoTest>
> --
> /var/tmp/portage/llvm-runtimes/offload-22.0.0.9999/work/offload/unittests/OffloadAPI/device/olGetDeviceInfo.cpp:14: Failure
> Parameterized test suite olGetDeviceInfoTest is defined via TEST_P, but never instantiated. None of the test cases will run. Either no INSTANTIATE_TEST_SUITE_P is provided or the only ones provided expand to nothing.
> 
> Ideally, TEST_P definitions should only ever be included as part of binaries that intend to use them. (As opposed to, for example, being placed in a library that may be linked in to get other utilities.)
> 
> To suppress this error for this test suite, insert the following line (in a non-header) in the namespace it is defined in:
> 
> GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST(olGetDeviceInfoTest);
> 
> 
> /var/tmp/portage/llvm-runtimes/offload-22.0.0.9999/work/offload/unittests/OffloadAPI/device/olGetDeviceInfo.cpp:14
> Parameterized test suite olGetDeviceInfoTest is defined via TEST_P, but never instantiated. None of the test cases will run. Either no INSTANTIATE_TEST_SUITE_P is provided or the only ones provided expand to nothing.
> 
> Ideally, TEST_P definitions should only ever be included as part of binaries that intend to use them. (As opposed to, for example, being placed in a library that may be linked in to get other utilities.)
> 
> To suppress this error for this test suite, insert the following line (in a non-header) in the namespace it is defined in:
> 
> GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST(olGetDeviceInfoTest);
> 
> 
> 
> ********************
> ```
> 
> and so on. The complete list is:
> 
> ```
> Failed Tests (39):
>   Offload-Unit :: OffloadAPI/./device.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetDeviceInfoSizeTest>
>   Offload-Unit :: OffloadAPI/./device.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetDeviceInfoTest>
>   Offload-Unit :: OffloadAPI/./event.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olCreateEventTest>
>   Offload-Unit :: OffloadAPI/./event.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olDestroyEventTest>
>   Offload-Unit :: OffloadAPI/./event.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetEventInfoSizeTest>
>   Offload-Unit :: OffloadAPI/./event.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetEventInfoTest>
>   Offload-Unit :: OffloadAPI/./event.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olSyncEventTest>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olCalculateOptimalOccupancy
> Test>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchKernelFooTest>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchKernelGlobalCtorTest>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchKernelGlobalDtorTest>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchKernelGlobalTest>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchKernelLocalMemReductionTest>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchKernelLocalMemStaticTest>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchKernelLocalMemTest>
>   Offload-Unit :: OffloadAPI/./kernel.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchKernelNoArgsTest>
>   Offload-Unit :: OffloadAPI/./memory.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olMemAllocTest>
>   Offload-Unit :: OffloadAPI/./memory.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olMemFillTest>
>   Offload-Unit :: OffloadAPI/./memory.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olMemFreeTest>
>   Offload-Unit :: OffloadAPI/./memory.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olMemcpyGlobalTest>
>   Offload-Unit :: OffloadAPI/./memory.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olMemcpyTest>
>   Offload-Unit :: OffloadAPI/./platform.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetPlatformInfoSizeTest>
>   Offload-Unit :: OffloadAPI/./platform.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetPlatformInfoTest>
>   Offload-Unit :: OffloadAPI/./program.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olCreateProgramTest>
>   Offload-Unit :: OffloadAPI/./program.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olDestroyProgramTest>
>   Offload-Unit :: OffloadAPI/./queue.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olCreateQueueTest>
>   Offload-Unit :: OffloadAPI/./queue.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olDestroyQueueTest>
>   Offload-Unit :: OffloadAPI/./queue.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetQueueInfoSizeTest>
>   Offload-Unit :: OffloadAPI/./queue.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetQueueInfoTest>
>   Offload-Unit :: OffloadAPI/./queue.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchHostFunctionKernelTest>
>   Offload-Unit :: OffloadAPI/./queue.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olLaunchHostFunctionTest>
>   Offload-Unit :: OffloadAPI/./queue.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olSyncQueueTest>
>   Offload-Unit :: OffloadAPI/./queue.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olWaitEventsTest>
>   Offload-Unit :: OffloadAPI/./symbol.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetSymbolGlobalTest>
>   Offload-Unit :: OffloadAPI/./symbol.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetSymbolInfoGlobalTest>
>   Offload-Unit :: OffloadAPI/./symbol.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetSymbolInfoKernelTest>
>   Offload-Unit :: OffloadAPI/./symbol.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetSymbolInfoSizeGlobalTest>
>   Offload-Unit :: OffloadAPI/./symbol.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetSymbolInfoSizeKernelTest>
>   Offload-Unit :: OffloadAPI/./symbol.unittests/GoogleTestVerification/UninstantiatedParameterizedTestSuite<olGetSymbolKernelTest>
> ```
> 
> …leaving 5 tests actually passing.

How are you configuring it? I think there might be something that goes wrong if you don't have all the plugin enabled and the tests don't handle it properly.

https://github.com/llvm/llvm-project/pull/156675