[Openmp-dev] OpenMP Offloading buildbot

[Michael Kruse via Openmp-dev]

Hi there,

I am working on adding a buildbot-worker to that tests offloading to
NVidia CPUs. This is work in-progress, but we could work on getting it
into a green state already before adding it to lab.llvm.org.

Currently, the worker is connected to my personal buildbot-master
which allows me to quickly apply changes. It comes in two
configurations:

openmp-offload-cuda-project
http://meinersbur.de:8011/#/builders/142

openmp-offload-cuda-runtime
http://meinersbur.de:8011/#/builders/143

The latter uses -DLLVM_ENABLE_RUMTIMES=openmp which uses the stage1
clang compiler to compile OpenMP. The worker intentionally has CUDA
not installed in a standard directory, so the option
-DCUDA_TOOLKIT_ROOT_DIR has to be used to find the CUDA SDK. This is
often found in a "module load" environment where multiple CUDA SDK
might be available to choose from.

So far I have identified the following issues for openmp-offload-cuda-project:

1. CMake: LIBOMPTARGET: Disabling tests using CUDA plugin as cuda may
not be available

This might be the result of CUDA not found in a default path. Tests
should use the CUDA found by CMake (CUDA_TOOLKIT_ROOT_DIR) and passed
to clang in the tests with --cuda-path. I tried to fix this previously
with https://reviews.llvm.org/D89974, but making the environment where
clang is built influence how it behaves or might be surprising,
especially when users expect to switch CUDA the version to use using
"module load". On the other side, regression tests should not depend
on the current "module load" environment.


2. CMake: LIBOMPTARGET: Not building NVPTX deviceRTL: clang not found

deviceRTLs/nvptx/CMakeLists.txt expect either
LIBOMPTARGET_NVPTX_CUDA_COMPILER to be set to a pre-compiled clang or
the project iself being compiled with clang. It should be capable of
using the clang that is built in the same project. When using
LLVM_ENABLE_RUMTIMES, these are the same.


3. Failing test: libomptarget::parallel_offloading_map.cpp

Assertion parallel_offloading_map.cpp:35: int main(int, char **):
Assertion `array[i] == ref' failed.


4. Failing test: libomptarget::memory_manager.cpp

Assertion memory_manager.cpp:37: int main(int, char **): Assertion
`buffer[j] == i' failed.



As far as I am aware this will be the first official builder that
tests OpenMP offloading. The only other worker for OpenMP gribozavr4
(https://lab.llvm.org/buildbot/#/workers/53) does not have libelf-dev
installed and therefore not even test CPU offloading for the following
builder:

openmp-clang-x86_64-linux-debian
openmp-clang-x86_64-linux-debian-release
openmp-gcc-x86_64-linux-debian


Further ideas for
openmp-offload-cuda-project/openmp-offload-cuda-runtime are to also
compile the SOLLVE V&V test suite
(https://github.com/SOLLVE/sollve_vv) but without it making fail the
build because not all features it tests have been implemented.


Michael