[Openmp-dev] AMDGPU and clang-tools-extra
Itaru Kitayama via Openmp-dev
openmp-dev at lists.llvm.org
Wed May 19 22:10:29 PDT 2021
Reducing the targets to only two (X86 and NVPTX) did not help either.
On Thu, May 20, 2021 at 1:59 PM Itaru Kitayama <itaru.kitayama at gmail.com> wrote:
>
> ```
> cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/tmp
> -DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_80
> -DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES=80 -DLLVM_ENABLE_PROJECTS="clang"
> -DLLVM_ENABLE_RUNTIMES=openmp
> -DLLVM_TARGETS_TO_BUILD="X86;NVPTX;AMDGPU"
> $SCRATCH/kitayama1/projects/llvm-project/llvm
> ```
>
> On Thu, May 20, 2021 at 1:58 PM Itaru Kitayama <itaru.kitayama at gmail.com> wrote:
> >
> > Jon,
> >
> > With this CMake configuration, build fails on my X86 environment.
> >
> > -- Configuring done
> > -- Generating done
> > CMake Warning:
> > Manually-specified variables were not used by the project:
> >
> > COMPILER_RT_BUILD_BUILTINS
> > LLVM_CONFIG_PATH
> > LLVM_DEFAULT_TARGET_TRIPLE
> > LLVM_ENABLE_PER_TARGET_RUNTIME_DIR
> > LLVM_ENABLE_PROJECTS_USED
> > LLVM_HOST_TRIPLE
> >
> >
> > -- Build files have been written to: /tmp/llvm/runtimes/runtimes-bins
> > [451/454] Performing build step for 'runtimes'
> > [142/173] Linking CXX shared library openmp/libomptarget/libomptarget.so
> > FAILED: openmp/libomptarget/libomptarget.so
> > : && /tmp/llvm/./bin/clang++ --target=x86_64-unknown-linux-gnu -fPIC
> > -fPIC -fvisibility-inlines-hidden -Werror=date-time
> > -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter
> > -Wwrite-strings -Wcast-qual -Wmissing-field-initializers
> > -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type
> > -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override
> > -Wno-comment -Wstring-conversion -Wmisleading-indentation
> > -fdiagnostics-color -ffunction-sections -fdata-sections -Wall
> > -Wcast-qual -Wformat-pedantic -Wimplicit-fallthrough -Wsign-compare
> > -Wno-extra -Wno-pedantic -std=gnu++14 -O3 -DNDEBUG -Wl,-z,defs
> > -Wl,-z,nodelete -shared -Wl,-soname,libomptarget.so -o
> > openmp/libomptarget/libomptarget.so
> > openmp/libomptarget/src/CMakeFiles/omptarget.dir/api.cpp.o
> > openmp/libomptarget/src/CMakeFiles/omptarget.dir/device.cpp.o
> > openmp/libomptarget/src/CMakeFiles/omptarget.dir/interface.cpp.o
> > openmp/libomptarget/src/CMakeFiles/omptarget.dir/rtl.cpp.o
> > openmp/libomptarget/src/CMakeFiles/omptarget.dir/omptarget.cpp.o
> > /tmp/llvm/lib/libLLVMSupport.a -ldl
> > -Wl,--version-script=/p/scratch/cjzam11/kitayama1/projects/llvm-project/openmp/libomptarget/src/exports
> > -lrt -ldl -lpthread -lm
> > /p/software/jurecadc/stages/2020/software/zlib/1.2.11-GCCcore-9.3.0/lib/libz.so
> > /p/software/jurecadc/stages/2020/software/ncurses/6.2-GCCcore-9.3.0/lib/libtinfo.so
> > /tmp/llvm/lib/libLLVMDemangle.a && :
> > /p/software/jurecadc/stages/2020/software/binutils/2.34-GCCcore-9.3.0/bin/ld:
> > /tmp/llvm/lib/libLLVMSupport.a(VirtualFileSystem.cpp.o): in function
> > `std::_Sp_counted_ptr_inplace<(anonymous namespace)::RealFSDirIter,
> > std::allocator<(anonymous namespace)::RealFSDirIter>,
> > (__gnu_cxx::_Lock_policy)2>::_M_get_deleter(std::type_info const&)':
> > VirtualFileSystem.cpp:(.text._ZNSt23_Sp_counted_ptr_inplaceIN12_GLOBAL__N_113RealFSDirIterESaIS1_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info+0x13):
> > undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info
> > const&)'
> > /p/software/jurecadc/stages/2020/software/binutils/2.34-GCCcore-9.3.0/bin/ld:
> > /tmp/llvm/lib/libLLVMSupport.a(VirtualFileSystem.cpp.o): in function
> > `std::_Sp_counted_ptr_inplace<(anonymous
> > namespace)::CombiningDirIterImpl, std::allocator<(anonymous
> > namespace)::CombiningDirIterImpl>,
> > (__gnu_cxx::_Lock_policy)2>::_M_get_deleter(std::type_info const&)':
> > VirtualFileSystem.cpp:(.text._ZNSt23_Sp_counted_ptr_inplaceIN12_GLOBAL__N_120CombiningDirIterImplESaIS1_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info+0x13):
> > undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info
> > const&)'
> > /p/software/jurecadc/stages/2020/software/binutils/2.34-GCCcore-9.3.0/bin/ld:
> > /tmp/llvm/lib/libLLVMSupport.a(VirtualFileSystem.cpp.o): in function
> > `std::_Sp_counted_ptr_inplace<llvm::vfs::(anonymous
> > namespace)::InMemoryDirIterator, std::allocator<llvm::vfs::(anonymous
> > namespace)::InMemoryDirIterator>,
> > (__gnu_cxx::_Lock_policy)2>::_M_get_deleter(std::type_info const&)':
> > VirtualFileSystem.cpp:(.text._ZNSt23_Sp_counted_ptr_inplaceIN4llvm3vfs12_GLOBAL__N_119InMemoryDirIteratorESaIS3_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info+0x13):
> > undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info
> > const&)'
> > /p/software/jurecadc/stages/2020/software/binutils/2.34-GCCcore-9.3.0/bin/ld:
> > /tmp/llvm/lib/libLLVMSupport.a(VirtualFileSystem.cpp.o): in function
> > `std::_Sp_counted_ptr_inplace<RedirectingFSDirRemapIterImpl,
> > std::allocator<RedirectingFSDirRemapIterImpl>,
> > (__gnu_cxx::_Lock_policy)2>::_M_get_deleter(std::type_info const&)':
> > VirtualFileSystem.cpp:(.text._ZNSt23_Sp_counted_ptr_inplaceI29RedirectingFSDirRemapIterImplSaIS0_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info[_ZNSt23_Sp_counted_ptr_inplaceI29RedirectingFSDirRemapIterImplSaIS0_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info]+0x13):
> > undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info
> > const&)'
> > /p/software/jurecadc/stages/2020/software/binutils/2.34-GCCcore-9.3.0/bin/ld:
> > /tmp/llvm/lib/libLLVMSupport.a(VirtualFileSystem.cpp.o): in function
> > `std::_Sp_counted_ptr_inplace<llvm::vfs::RedirectingFSDirIterImpl,
> > std::allocator<llvm::vfs::RedirectingFSDirIterImpl>,
> > (__gnu_cxx::_Lock_policy)2>::_M_get_deleter(std::type_info const&)':
> > VirtualFileSystem.cpp:(.text._ZNSt23_Sp_counted_ptr_inplaceIN4llvm3vfs24RedirectingFSDirIterImplESaIS2_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info[_ZNSt23_Sp_counted_ptr_inplaceIN4llvm3vfs24RedirectingFSDirIterImplESaIS2_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info]+0x13):
> > undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info
> > const&)'
> > /p/software/jurecadc/stages/2020/software/binutils/2.34-GCCcore-9.3.0/bin/ld:
> > /tmp/llvm/lib/libLLVMSupport.a(VirtualFileSystem.cpp.o):VirtualFileSystem.cpp:(.text._ZNSt23_Sp_counted_ptr_inplaceIN4llvm3vfs6detail15RecDirIterStateESaIS3_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info[_ZNSt23_Sp_counted_ptr_inplaceIN4llvm3vfs6detail15RecDirIterStateESaIS3_ELN9__gnu_cxx12_Lock_policyE2EE14_M_get_deleterERKSt9type_info]+0x13):
> > more undefined references to
> > `std::_Sp_make_shared_tag::_S_eq(std::type_info const&)' follow
> > clang-13: error: linker command failed with exit code 1 (use -v to see
> > invocation)
> > [172/173] Building CXX object openmp/r.../CMakeFiles/omp.dir/kmp_settings.cpp.o
> > ninja: build stopped: subcommand failed.
> > FAILED: runtimes/runtimes-stamps/runtimes-build
> > cd /tmp/llvm/runtimes/runtimes-bins &&
> > /p/software/jurecadc/stages/2020/software/CMake/3.18.0-GCCcore-9.3.0/bin/cmake
> > --build .
> > ninja: build stopped: subcommand failed.
> >
> > On Thu, May 20, 2021 at 11:38 AM Ye Luo <xw111luoye at gmail.com> wrote:
> > >
> > > Based on the CMake output, LLVM was found on the machine so LLVM_DIR is set.
> > > However, clang was not found so CLANG_TOOL is NOT-FOUND
> > >
> > > if (LLVM_DIR)
> > > # Builds that use pre-installed LLVM have LLVM_DIR set.
> > > find_program(CLANG_TOOL clang PATHS ${LLVM_TOOLS_BINARY_DIR} NO_DEFAULT_PATH)
> > > find_program(LINK_TOOL llvm-link PATHS ${LLVM_TOOLS_BINARY_DIR}
> > > NO_DEFAULT_PATH)
> > > find_program(OPT_TOOL opt PATHS ${LLVM_TOOLS_BINARY_DIR} NO_DEFAULT_PATH)
> > > libomptarget_say("Building AMDGCN device RTL. Using clang: ${CLANG_TOOL}")
> > >
> > > Use ENABLE_RUNTIMES does workaround this issue as this part is bypassed.
> > >
> > > @Jon please take care of the case when any of the find_program fails.
> > >
> > > Second issue is libelf. I had this issue in the past when libelf is spack installed.
> > > libomptarget CMake found libelf library files but didn't check the header files.
> > > This caused all the complaints about <libelf/libelf.h> not found.
> > > The workaround in my case is apt instal libelf-dev.
> > >
> > > I think the libelf search needs to be more robust in libomptarget.
> > >
> > > Best,
> > > Ye
> > > ===================
> > > Ye Luo, Ph.D.
> > > Computational Science Division & Leadership Computing Facility
> > > Argonne National Laboratory
> > >
> > >
> > > On Wed, May 19, 2021 at 8:19 PM Itaru Kitayama <itaru.kitayama at gmail.com> wrote:
> > >>
> > >> Hi all,
> > >> As the files are too large for the list; I've uploaded them to:
> > >>
> > >> https://github.com/spack/spack/issues/23654
> > >>
> > >> On Thu, May 20, 2021 at 10:14 AM Itaru Kitayama
> > >> <itaru.kitayama at gmail.com> wrote:
> > >> >
> > >> > Hi Ye,
> > >> > https://github.com/ikitayama/spack for-ye , as I need to set
> > >> > ENABLE_RUNTIMES explicitly.
> > >> >
> > >> > $ spack install llvm at main+omp_debug+cuda cuda_arch=80
> > >> >
> > >> > On Thu, May 20, 2021 at 6:53 AM Ye Luo <xw111luoye at gmail.com> wrote:
> > >> > >
> > >> > > -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_ENABLE_RUNTIMES="openmp"
> > >> > > I think this is what Jon refers to.
> > >> > >
> > >> > > If you have issues building llvm at main in spack. Please provide your spack command and attach the following files
> > >> > > spack-build-01-cmake-out.txt
> > >> > > spack-build-02-build-out.txt
> > >> > > from your spack-stage llvm build folder.
> > >> > > Best,
> > >> > > Ye
> > >> > > ===================
> > >> > > Ye Luo, Ph.D.
> > >> > > Computational Science Division & Leadership Computing Facility
> > >> > > Argonne National Laboratory
> > >> > >
> > >> > >
> > >> > > On Wed, May 19, 2021 at 4:34 PM Itaru Kitayama <itaru.kitayama at gmail.com> wrote:
> > >> > >>
> > >> > >> Jon, Ye,
> > >> > >> Is this
> > >> > >>
> > >> > >> -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_ENABLE_RUNTIMES="openmp"
> > >> > >>
> > >> > >> correct? I used, and Spack as well, combined them (clang and openmp
> > >> > >> projects) together indeed in ENABLE_PROJECTS.
> > >> > >> On Thu, May 20, 2021 at 5:18 AM Ye Luo via Openmp-dev
> > >> > >> <openmp-dev at lists.llvm.org> wrote:
> > >> > >> >
> > >> > >> > spack install llvm at main does work on my workstation with ubuntu 20.04. rocm is installed and amd pieces for OpenMP offload are rebuilt properly. I have limited ideas about the experience on other OSes.
> > >> > >> >
> > >> > >> > yeluo at epyc-server:~/opt/llvm-clang/llvm-project$ ls /home/packages/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-main-6n25gihrin6rc727fbothf44yq6zbp5a/lib/libomp
> > >> > >> > libomp.so libomptarget-amdgcn-gfx803.bc libomptarget.rtl.cuda.so
> > >> > >> > libomptarget-amdgcn-gfx700.bc libomptarget-amdgcn-gfx900.bc libomptarget.rtl.x86_64.so
> > >> > >> > libomptarget-amdgcn-gfx701.bc libomptarget-amdgcn-gfx906.bc libomptarget.so
> > >> > >> > libomptarget-amdgcn-gfx801.bc libomptarget.rtl.amdgpu.so
> > >> > >> >
> > >> > >> > yeluo at epyc-server:~/opt/llvm-clang/llvm-project$ which clang-tidy
> > >> > >> > /home/packages/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-main-6n25gihrin6rc727fbothf44yq6zbp5a/bin/clang-tidy
> > >> > >> >
> > >> > >> > spack uses ENABLE_PROJECTS and doesn't use ENABLE_RUNTIMES right now.
> > >> > >> > So the output from building clang-tools-extra and openmp can be blended.
> > >> > >> >
> > >> > >> > Best,
> > >> > >> > Ye
> > >> > >> > ===================
> > >> > >> > Ye Luo, Ph.D.
> > >> > >> > Computational Science Division & Leadership Computing Facility
> > >> > >> > Argonne National Laboratory
> > >> > >> >
> > >> > >> >
> > >> > >> > On Wed, May 19, 2021 at 2:22 PM Jon Chesterfield via Openmp-dev <openmp-dev at lists.llvm.org> wrote:
> > >> > >> >>>
> > >> > >> >>> Date: Wed, 19 May 2021 15:09:18 +0900
> > >> > >> >>> From: Itaru Kitayama via Openmp-dev <openmp-dev at lists.llvm.org>
> > >> > >> >>> To: openmp-dev <openmp-dev at lists.llvm.org>
> > >> > >> >>> Subject: [Openmp-dev] AMDGPU and clang-tools-extra
> > >> > >> >>> Message-ID:
> > >> > >> >>> <CANW9uysd4uCQ7j3Jr1qmn2=RPo5a5OGU9e8-vPuuGv=P0nAXDQ at mail.gmail.com>
> > >> > >> >>> Content-Type: text/plain; charset="UTF-8"
> > >> > >> >>>
> > >> > >> >>> I am trying to build llvm on main branch with Spack with help from Tom, however
> > >> > >> >>> the AMDGPU code does not seem to work well with clang-tools-extra
> > >> > >> >>> project, which
> > >> > >> >>> is added together with the clang project when a Spack build initiated.
> > >> > >> >>> Has anyone
> > >> > >> >>> observed an issue with AMDGPU code with tools in upstream?
> > >> > >> >>
> > >> > >> >>
> > >> > >> >> None reported. I don't build clang-tools-extra, seems totally independent.
> > >> > >> >>
> > >> > >> >> Are you building openmp with ENABLE_PROJECTS and a clang that can't build amdgpu code, instead of ENABLE_RUNTIMES? If so, use ENABLE_RUNTIMES, a working clang, or disable the amdgpu library.
> > >> > >> >>
> > >> > >> >> Otherwise, please expand on 'does not seem to work well'.
> > >> > >> >>
> > >> > >> >> Jon
> > >> > >> >> _______________________________________________
> > >> > >> >> Openmp-dev mailing list
> > >> > >> >> Openmp-dev at lists.llvm.org
> > >> > >> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev
> > >> > >> >
> > >> > >> > _______________________________________________
> > >> > >> > Openmp-dev mailing list
> > >> > >> > Openmp-dev at lists.llvm.org
> > >> > >> > https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev
More information about the Openmp-dev
mailing list