[libcxx-commits] [libcxx] [libcxxabi] [llvm] Adding Separate OpenMP Offloading Backend to `libcxx/include/__algorithm/pstl_backends` (PR #66968)

[Vedant Paranjape via libcxx-commits]

VedantParanjape wrote:

> Hello @AntonRydahl , I am having a difficult time to compile the tests in the PR.
> 
> I build this PR code with the following command
> 
> ```
> cmake -G Ninja -DCMAKE_BUILD_TYPE=Release \
>           -DCMAKE_C_COMPILER=gcc \
>           -DCMAKE_CXX_COMPILER=g++ \
>           -DCMAKE_INSTALL_PREFIX="$INSTALLDIR" \
>           -DLIBCXX_ENABLE_WERROR=YES \
>           -DLIBCXXABI_ENABLE_WERROR=YES \
>           -DLIBUNWIND_ENABLE_WERROR=YES \
>           -DLIBCXX_ENABLE_CLANG_TIDY=ON \
>           -DLLVM_LIT_ARGS="-sv --xunit-xml-output test-results.xml --timeout=1500 --time-tests" \
>           -DLLVM_ENABLE_PROJECTS="clang;lld;" -DCLANG_DEFAULT_LINKER="lld" -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind;offload;pstl;openmp" -DLIBCXX_PSTL_BACKEND="openmp" -DLLVM_TARGETS_TO_BUILD="X86;AMDGPU" \
>           -DPSTL_PARALLEL_BACKEND="omp" -DLIBCXX_CXX_ABI=libcxxabi -DLIBCXX_ENABLE_THREADS=ON \
>                 $CLANG_ROOT/llvm-project/llvm
> ```
> 
> I am trying to compile of the tests with the following command:
> 
> ```
> clang++  -Wl,-rpath,../clang/build/lib  -I ../clang/build/projects/runtimes/src/  -I ../clang/build/runtimes/runtimes-bins/openmp/runtime/src/  -fopenmp -fexperimental-library -fopenmp-targets=amdgcn-amd-amdhsa test.cpp -o test
> ```
> 
> however, it throws an error about tbb, which is interesting, since i specifically compiled for the Openmp backend.
> 
> ```
> ld.lld: error: undefined symbol: tbb::interface7::internal::isolate_within_arena(tbb::interface7::internal::delegate_base&, long)
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(__pstl::execution::v1::parallel_unsequenced_policy const& tbb::interface7::internal::isolate_impl<void, void __pstl::__tbb_backend::__parallel_for<__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, void __pstl::__internal::__pattern_walk1<__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>>(__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::'lambda'()::operator()() const::'lambda'(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>)>(__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0)::'lambda'() const>(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>&))
> 
> ld.lld: error: undefined symbol: tbb::task_group_context::~task_group_context()
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(tbb::interface9::internal::start_for<tbb::blocked_range<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>>, __pstl::__tbb_backend::__parallel_for_body<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, void __pstl::__internal::__pattern_walk1<__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>>(__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::'lambda'()::operator()() const::'lambda'(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>)>, tbb::auto_partitioner const>::run(tbb::blocked_range<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>> const&, __pstl::__tbb_backend::__parallel_for_body<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, void __pstl::__internal::__pattern_walk1<__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>>(__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::'lambda'()::operator()() const::'lambda'(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>)> const&, tbb::auto_partitioner const&))
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(tbb::interface9::internal::start_for<tbb::blocked_range<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>>, __pstl::__tbb_backend::__parallel_for_body<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, void __pstl::__internal::__pattern_walk1<__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>>(__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::'lambda'()::operator()() const::'lambda'(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>)>, tbb::auto_partitioner const>::run(tbb::blocked_range<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>> const&, __pstl::__tbb_backend::__parallel_for_body<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, void __pstl::__internal::__pattern_walk1<__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>>(__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::'lambda'()::operator()() const::'lambda'(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>)> const&, tbb::auto_partitioner const&))
> 
> ld.lld: error: undefined symbol: tbb::task_group_context::init()
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(tbb::task_group_context::task_group_context(tbb::task_group_context::kind_type, unsigned long))
> 
> ld.lld: error: undefined symbol: tbb::internal::allocate_root_with_context_proxy::allocate(unsigned long) const
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(operator new(unsigned long, tbb::internal::allocate_root_with_context_proxy const&))
> 
> ld.lld: error: undefined symbol: tbb::internal::allocate_root_with_context_proxy::free(tbb::task&) const
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(operator delete(void*, tbb::internal::allocate_root_with_context_proxy const&))
> 
> ld.lld: error: undefined symbol: vtable for tbb::task
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(tbb::task::task())
> >>> the vtable symbol may be undefined because the class is missing its key function (see https://lld.llvm.org/missingkeyfunction)
> 
> ld.lld: error: undefined symbol: tbb::internal::get_initial_auto_partitioner_divisor()
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(tbb::interface9::internal::adaptive_mode<tbb::interface9::internal::auto_partition_type>::adaptive_mode())
> 
> ld.lld: error: undefined symbol: tbb::internal::allocate_child_proxy::allocate(unsigned long) const
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(tbb::interface9::internal::allocate_sibling(tbb::task*, unsigned long))
> 
> ld.lld: error: undefined symbol: tbb::internal::allocate_continuation_proxy::allocate(unsigned long) const
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(operator new(unsigned long, tbb::internal::allocate_continuation_proxy const&))
> 
> ld.lld: error: undefined symbol: tbb::internal::allocate_continuation_proxy::free(tbb::task&) const
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(operator delete(void*, tbb::internal::allocate_continuation_proxy const&))
> 
> ld.lld: error: undefined symbol: tbb::task_group_context::is_group_execution_cancelled() const
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(tbb::task::is_cancelled() const)
> 
> ld.lld: error: undefined symbol: typeinfo for tbb::task
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(typeinfo for tbb::interface9::internal::start_for<tbb::blocked_range<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>>, __pstl::__tbb_backend::__parallel_for_body<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, void __pstl::__internal::__pattern_walk1<__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>>(__pstl::execution::v1::parallel_unsequenced_policy const&, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, main::$_0, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::'lambda'()::operator()() const::'lambda'(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int>>>)>, tbb::auto_partitioner const>)
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(typeinfo for tbb::interface9::internal::flag_task)
> 
> ld.lld: error: undefined symbol: tbb::task::note_affinity(unsigned short)
> >>> referenced by test5.cpp
> >>>               /var/tmp/singhal2/test5-0eaed1.o:(vtable for tbb::interface9::internal::flag_task)
> /usr/bin/clang-linker-wrapper: error: 'ld.lld' failed
> clang++: error: linker command failed with exit code 1 (use -v to see invocation)
> ```
> 
> I can fix this error if i add `-ltbb` to the end of the compile command. But then it is never offloaded. although if i add save temps i do see files compiled for the amd gpu target.
> 
> Do you have any clue what is happening here?

Ping! could someone help with this!

https://github.com/llvm/llvm-project/pull/66968