<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/90539>90539</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            icpx: error: ptxas command failed with exit code 255
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          J-shel
      </td>
    </tr>
</table>

<pre>
    I'm trying to install cloudsc_sycl on Nvidia A100 GPU using OneAPI DPC++ compiler. 

First, I download cloudsc_sycl from [GitHub - ecmwf-ifs/dwarf-p-cloudsc at nams_pvc_sycl](https://github.com/ecmwf-ifs/dwarf-p-cloudsc/tree/nams_pvc_sycl)

Then I installed OneAPI for Nvidia GPU following this website: [Install oneAPI for NVIDIA GPUs - Guides - oneAPI for NVIDIA® GPUs - Products - Codeplay Developer](https://developer.codeplay.com/products/oneapi/nvidia/2023.0.0/guides/get-started-guide-nvidia.html)


Below is my environment:
-----------------------------------
CUDA Version: 12.4
OneAPI: l_BaseKit_p_2024.1.0.596_offline.sh and l_HPCKit_p_2024.1.0.560_offline.sh
DPC++ plugin: oneapi-for-nvidia-gpus-2024.1.0-cuda-12.0-linux.sh
-----------------------------------


Following are some settings in env.sh and toolchain.cmake under dwarf-p-cloudsc-nams/arch/ecmwf/hpc2020/intel-sycl/2024.1 dir:
-----------------------------------
env.sh:
export IntelSYCL_DIR="/opt/intel/oneapi/compiler/2024.1/lib/cmake/IntelSYCL"
export HDF5_ROOT="/usr/local/cloudsc_sycl_libs"


toolchain.cmake:
set(CMAKE_CXX_COMPILER "/opt/intel/oneapi/compiler/2024.1/bin/icpx")
set(CMAKE_C_COMPILER "/opt/intel/oneapi/compiler/2024.1/bin/icx")
set(CMAKE_Fortran_COMPILER "/opt/intel/oneapi/compiler/2024.1/bin/ifort")

set(CMAKE_C_FLAGS "-O3 -L/opt/intel/oneapi/compiler/2024.1/lib -qopenmp -fgpu-rdc")
set(CMAKE_CXX_FLAGS "-O3 -L/opt/intel/oneapi/compiler/2024.1/lib -qopenmp -lstdc++ -fgpu-rdc")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsycl-early-optimizations -fsycl")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsycl-targets=nvptx64-nvidia-cuda")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Xsycl-target-backend --cuda-gpu-arch=sm_80")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -I/opt/intel/oneapi/compiler/2024.1/include -I/usr/local/cuda/include")
-----------------------------------


Then I started to install cloudsc_sycl by running "./cloudsc-bundle build --clean --build-dir=build-sycl --with-gpu --with-sycl --arch=arch/ecmwf/hpc2020/intel-sycl/2024.1 -j 1" and met the following error:
-----------------------------------
[ 69%] Building CXX object cloudsc-dwarf/src/cloudsc_sycl/CMakeFiles/dwarf-cloudsc-scc-sycl-lib.dir/cloudsc/mycpu.cpp.o
icpx: warning: -Z-reserved-lib-stdc++: 'linker' input unused [-Wunused-command-line-argument]
[ 70%] Linking CXX shared library ../../../lib/libdwarf-cloudsc-scc-sycl-lib.so
ptxas fatal : Unresolved extern function 'fmin'
llvm-foreach:
icpx: error: ptxas command failed with exit code 255 (use -v to see invocation)
Intel(R) oneAPI DPC++/C++ Compiler 2024.1.0 (2024.1.0.20240308)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/intel/oneapi/compiler/2024.1/bin/compiler
Configuration file: /opt/intel/oneapi/compiler/2024.1/bin/compiler/../icpx.cfg
icpx: note: diagnostic msg: Error generating preprocessed source(s).
make[2]: *** [cloudsc-dwarf/src/cloudsc_sycl/CMakeFiles/dwarf-cloudsc-scc-sycl-lib.dir/build.make:147: lib/libdwarf-cloudsc-scc-sycl-lib.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:1335: cloudsc-dwarf/src/cloudsc_sycl/CMakeFiles/dwarf-cloudsc-scc-sycl-lib.dir/all] Error 2
make: *** [Makefile:166: all] Error 2
-----------------------------------

I have seen a similar issue on this website "https://github.com/llvm/llvm-project/issues/65403", but adding "-fgpu-rdc" didn't work in my case. Any suggestions?

Thanks!
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJy0WNty2zgS_RropQssChR1edCDLmFGO87alUlmvfuiAskmhTEIcAFQsvbrt8CLLHud7NjJpFI0TKBPdx80Dprm1opSIS5JvCbxdsQbd9Bm-TdqDyhHqc7Pyx1hswqcOQtVgtMglHVcSsikbnKb7e05k6AV_P0ocsFhNQ5D-Hj3FRrrDW4Vru52sL3bELYmbA2Zrmoh0QRAwi0JV90zEcY6wjawg1yflNQ8f-6gMLoCEq8_CvdLkwIFzKpTQUVhCUvyEzcFrWlvAdyB4pXd18fOmsRbwuYH52pLohVhCWFJKdyhSYNMV4Ql3wEjLHEGkbDkOSRbXMf_5YAKdgM3mA95F9oMxHhOCi2lPrVEHoSFE6ZWOCTRyqe264nVV6a_77a7lTe1QOFjI3L0g_9ZQTaMrD4M6-6MzpvM-eFG51hLfoYtHlHqGs1rXOTDZJD163te6h6JsEQr5LXwNLTpEJawkEVBGISezDYyP0BHrePGYU7bl7RbHhxc9ZKz7rlGqU8gLFRnQHUURqsKlfOxtfP0___rFm6-blfwOxortPKMjlkw6Wa6vfDv5H7NLf4q3L7es5BNgnEQBvFiutdFIYXCwB6Aqxzk_pe7zctl0_BqWYf8VNa1bErR-u2IooU2fe60rBtLBxyaNTmnYxaEVArVPF6w_nSiz87NpaC4QbC6QrDonFClBaE8oUNKTmuZHbhQQVbxB4RG5WjgRbFTX-OEJdxkh-FYEJYc6oyFzG-0UA4l7Q5A0qUEuTBv360usosdPtbaONh5-N_-ubnZb3efSbQljPnSq93g-roQByW5REJYIkXqZ3yGhCUXOI9z7eeXbRLvP9_efrn4aKzHkTrj3se19OylSO0F4Pr5gtJLMhYdYfPNp9WvH_ab-_v95vbT3e7mw2d4czqpUH5pVj-2potX8H8C-jfBE22c4erHXRTauGsnr-WR3Kw-_uY90NsI6M2b9x3ov3WNqqqBFmXdUJNn3yTt_v5nupPWeV-tDrzNN2ETMlu_mCCzLdDClx5FbuSZ6tqJSvyHO6GV7ad-CrzjpkRnSbRVx9o9TieDYnmN-kEP91ceaMqzB1Q50E79PEWtxERbW-3n4Q-62r1t84TKZJNjZ_fi3Ld5Dyuuw3qXOvddQX8hfrN5Ss9gGqW8ihPGgif9oWmjcomQNkK25EnkCihtf6et7m67cYtD6Um4g2d3GPave67fpOr0DxgTxtq7o0IH7oBX7Qsao9-h-iRew3RBWEziLax95B5sc38POv0DMzfwQtt7ibDEmuyFHBOWbD7xB0yExKd2bbCzWdbmQaVIA0_QxZiwpDpndRNkdR3oLpxWWKMVnLjx7Psh_Rc1aNEcMfcY9Olot10am0mhHnw9zUCounHQqMZi7vs3-o9uTDNdVVx5e4WUm7JpG5p4-8TBLOw5uBHqYaDAHrjBHKRIDTdnCHwhPD26m02K9DsJ2z6v2j1yCwV3XIIP-6syaLU8Yg746NAoKBqVeTnxGRWVF-lZZyrlsfK9C_Ls6XYeeBp2HToHfZ5QcOFbXl9xgI_Cge8igcUxEDZvLAI9-tq3iCDUUWetkF1O1q47s_PPhC2GzvbSV_nd7pV10x9lGDopj37pzvwgjML5BfZLqzw-2Mf5dD-d0EY9KH1SXddFS9UMZ9Qgz6HSOco2NW3F4xBZ381v2w4H3nP1XWa6HlWrQpSNaRmAQkj8Ydy-PPwWBVlRPt8wpbuvi1zwUmnrRAaVbev8g99KKFGhD0aVUBusjc7Q-mq2ujEZEja3hC2CDrNtceI184XcBr3q_vvS_wuObatrQd9XjSeztn3_c2cg3vbpjZ9FPn4t8mdB-bHfFOZdRlHsV_8FqXEpn2JkVzG-DG6Ix4cznfrp10zfeDXt4MCP_jMBFXCwohKSGxDWNug_468_TP119J2vZq8V_Q9aG-0F3FeiR_I8TONJGLV36AbSxgHP8_6Ku26SIBe5lx8HJ20e_EdLdYaMWwxgpc5gm7JE23Y-JEqeX65cPVjCxqN8GeWLaMFHuBzPxpNJtIjG8eiwnPMZn2VRzibFlBWTeBZFk2IaT_P5fByOWTwSy1Y2JlEYjsN5FAcZG2ezNI1mWZhO53xKJiFWXMjA5xhoU47a7JaLMI4WI8lTlLb90wljCk8diT7jeDsyy5aXtCktmYRSWGefUJxwEpfvldVRY-Ty_fvSBv_fAAAA__-T43vS">