<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/114210>114210</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [clang] clang driver fails to find headers/libraries when specified --gcc-install-dir or --gcc-toolchain or --sysroot to a cross-compiler directory
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            clang
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          lbmeng
      </td>
    </tr>
</table>

<pre>
    I am trying to use clang 18 to cross compile a program for Arm 32-bit target.
The clang 18 was installed from the official LLVM Ubuntu repository following the instructions [here](https://apt.llvm.org/).

The GCC toolchain I was using is from [bootlin](https://toolchains.bootlin.com/downloads/releases/toolchains/armv7-eabihf/tarballs/armv7-eabihf--glibc--stable-2024.02-1.tar.bz2).

The rough GCC toolchain directory layout is:
```
/path/to/bootlin/toolchain
├── arm-buildroot-linux-gnueabihf
│   ├── bin
│   ├── include
│   │   └── c++
│   ├── lib
│   └── sysroot
│       ├── bin
│       ├── dev
│       ├── etc
│       ├── lib
│       ├── lib32 -> lib
│       ├── media
│       ├── mnt
│       ├── opt
│       ├── proc
│       ├── root
│       ├── run
│       ├── sbin
│       ├── sys
│       ├── tmp
│       ├── usr
│       └── var
├── bin
├── etc
├── include
├── lib
│   ├── gcc
│   │   └── arm-buildroot-linux-gnueabihf
...
├── lib64 -> lib
├── libexec
└── share
```

Compiling a simple program using:
```
$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --gcc-toolchain=/path/to/bootlin/toolchain -c test.c -o test.o
clang-18: warning: argument unused during compilation: '--gcc-toolchain=/path/to/bootlin/toolchain' [-Wunused-command-line-argument]
test.c:1:10: fatal error: 'stdio.h' file not found
    1 | #include <stdio.h>
      |          ^~~~~~~~~
1 error generated.
```

clang-18 complains `--gcc-toolchain` is unused. Changing to use `--gcc-install-dir` does not make it better. The same unused argument warning was thrown. By looking at the [documentation](https://clang.llvm.org/docs/ClangCommandLineReference.html), the doc does not mention how exactly these 2 options affects what behavior of clang.

Changing to use`--sysroot` failed too. The sysroot value was obtained from the result of `/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-gcc --print-sysroot`.

Then I found a workaround to help locate the header file by creating a symbolic link `include` to point to `usr/include` in the sysroot directory when using 
```
/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot$ ln -s usr/include include
```

Then I removed `-c` to see if I can compile and link the program correctly, I got:
```
$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot test.c -o test.o
ld.lld: error: unable to find library -lc
ld.lld: error: unable to find library -lm
ld.lld: error: unable to find library -lgcc
clang-18: error: ld.lld command failed with exit code 1 (use -v to see invocation)
```

I realized the new clang compiler driver defaults to use ld.lld, so I added `-fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld` in the command line, which made clang to successfully locate libc and libm libraries, but not libgcc.

```
$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld test.c -o test.o
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: library search path "/usr/lib/llvm-18/lib/clang/18/lib/baremetal" is unsafe for cross-compilation
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: cannot find -lgcc: No such file or directory
clang-18: error: ld.lld command failed with exit code 1 (use -v to see invocation)
```

The libgcc.a is located in the bootlin gcc toolchain directorty here:
```
$ /path/to/bootlin/toolchain$ find . -name libgcc.a
./lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0/libgcc.a
```

So I added `-L/path/to/bootlin/toolchain/lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0` in the command line, then libgcc was located by clang, but the final linking procedure still complained:

```
$ clang-18 --target=arm-eabi -mcpu=cortex-a7 -mthumb -mfpu=neon -mfloat-abi=hard -Wall --sysroot=/path/to/bootlin/toolchain/arm-buildroot-linux-gnueabihf/sysroot -fuse-ld=/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld test.c -L/path/to/bootlin/toolchain/lib/gcc/arm-buildroot-linux-gnueabihf/12.3.0 -o test.o
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: library search path "/usr/lib/llvm-18/lib/clang/18/lib/baremetal" is unsafe for cross-compilation
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: warning: cannot find entry symbol _start; defaulting to 00010140
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(printf.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr1'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(getsysstats.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(getsysstats.o):(.ARM.exidx+0x8): undefined reference to `__aeabi_unwind_cpp_pr1'
/path/to/bootlin/toolchain/bin/arm-buildroot-linux-gnueabihf-ld: /path/to/bootlin/toolchain/bin/../arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libc.a(madvise.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
...
```

I gave up here. So what is the expected way of using clang to cross-compile a program and using libraries from a gcc cross-toolchain?
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzsWk1v4zjS_jXMpSBBouzEOfiQOO0XDfS8h5ndnWODIksWtylSICk7nsP-9gVJyXGyTuK0pzHAzBhpt2WpyKqH9fFUJcw5udGISzK_J_OHKzb41tilqjvUm6vaiP3yM7AOvN1LvQFvYHAIXDG9gXIRrrk1zgE3XS8VAoPemo1lHTTGwp3toKJZLT14Zjfoc1I8kOLuH-3RGjvmQGrnmVIooLGmA98imKaRXDIFX7786yf4Zz1oP4DF3jjpjd1DY5Qyu6hUi3EBO3AvjXZA5vctWiTzB0IXrfe9I9UdoWtC16z3uVLbLjd2E7-5HVV6Uuz_VivwxijeMqnhc9RvcGEj6ZJ6ZH5fG-OV1Ke2OMi6fHwq56YjdC3MTivDhCN0bVEhc-iePR_0s932JkNWy7YJ95itmVIv72TZRsmaZ5nzrFaY0YLO8oJmZe6Zzevf6Cm7rBk27QvrhLTII5yK7c3gQUY7kuR1Mf6kS7rumW-jwoSuJwCO9B-f-0TJ7YzcrsYPi-LwAZjtsnqQSlhjfKakHh6zjR5Gc4_FF5SsKLkr0ju8tWj9YucPiErN1SDwDPFXFp2dWJQTeh9-vlMnJetzRE_t7PYuAPuWeHhdAOZ74gK3l4ij55eIvwPdGeIVhYxUny5dqUMh2UUL6ItO0fQXiffWXHQOlzqhHS5yQnehE7u9u0Tcd_0l4oOz54qfygFbZt_Pwy8BOisYz86gl6S1U6Ib_qY3fjxPnlGG8jw_y6br2emE8drz-IgvUD2ZyFtm8XQVju-ryLYCJWHgZNcrPNCuyFRer-GzxLuycgFZljgZqR4CIMF6yDreD6R64MZ6fMzYDWSdb4euhqxr4h2NRocLZZjPWC1J9dAyKyD7lSkFWbbhPHtiBNXD-7QBMg4enc85ZCZ9MkndSVVS3cGOWZ0sA2Y3Q4faw6AHhwLEYAMUiYKyQAHDU4TefI82hN4Efpf9mhbPuOk6pkXwEsymnQPtixomvUl1V4Z_Rdi3YZ4pQGuNHdVwXkiTt2HlJnBkbTw0ZtAirREiugRyswJCqzGegFSrSaz69PQcxOcOLzL_9J_xlZ4p08awQY2WeRT5G1508ISAnAoEFMh18RK06yLw3oRGDquW6c1RJ3AQGBl8JqQNIsKgi4Z27BuC9FCj92hzCEzUsQ6nwzsc5njAkW771pqdzuF-D8qYb9HRfeT5ZH4vDI8S6aRPEPBo1zHLF4YHBr0K36_SeX6RGn_GBi1qjnnrO0XoLaGruIkw_MgA1GEjaM0O8JFxr_bhIYdAQ52NDQdrGuTewa5lwdKWbaWxYJoUbM-Y-AsAI34Tb7suoGEydEHemBGqdAu2TA0YsTG1Z1Ifd0oW3aB82C4c8Bkevq7j-5tpMBwqZFlvpfZHCr7sKkKDFH0ZGOyM_cZsvPAGWlQ9KMOZx6hli0ygTRFQ74FbZH7MYPuuNkpyUFJ_CzZMReW6CAv1RmofPpDrIhRHuj66L3VcfILpqZ_ZBd1S2_a97cw7CBG6nnChM1AaMgfP9HtRHE-F4Iigxc5sUcRo4qPZDhFkA5-BM_3UXmuRUApGTymfGxusVvvgv59hY_wfmP8nSM7Lteci_EqBUCJXSoQ0e8i3gw4dcQCwkRGs2jK7h0zxj4t0Hxc5cJXj2nUQSyvBWFOmYN9J3wI-Sg_cCAy1gC5Cbs22Bz_Q2xBIId_R2zfcKXgSU_K3kEFaBI27ccwy-o8FYeU2_IcNG5R3UxofTaQrcAY-AxNi9MZmcJgF4887znMSixJHgTtBEcpr2H7XSt5Cx8Q0IAoIDJyjc82g1H7KKErWfIyGuhvxl-jCEvXgY-JWst5w_ixh_WnC4cecyytR9nue_HMmN8WNQ2Z5C2EXIJQSuk6JNLBqug6VPETSdB2PjND10Vc1s9ihZ4pQmuiKYw3GGWScT2bH5PBHGMWZjrQuZIOUBao7-P_ou22qecY-Vac_MEUEUjFGBgtIpXgSUzyOOISe68SY0O8hDlbfKC_n4DpLOOWQ6cAEJ3XGzutwqAHF90OipHmVF0noaJmTxv_yPLl9OcsJvkOZ1_ObDwU_aRrZ3AR_YETJrVMCC7KN1EzFch9YTG8NRzFYBOelUgfSjuLpOP5Och9Icj_q9P9OnxcZdZxJUftgXuwP4KvzzHpS3U_sZeyiiqIoi3L2EV5_vmIfWC7PP9IxPD8iWYfERRex3WpyE3J4aGcX-d3PP-X4KMUjoffFY5FuwKAFNrELtFMTO3ZIX7-ysNnXQe-kFl9533_tbUnozZ8Bng16t3fOM-9-d4yKvxJGi7-2H3VMbKXDH-hDT_Pj033ahm0Rhj7SqRx-MWl0JF0s_PjYIw-sYMf2YJpxjHHoh45T8vFv3APPSI8e-qE0JGKRziWxo2no-kosK3Fb3bIrXJY3VUEXi7Iqr9rlomlENZuV8_p2Jlh5i_Oa37Cywqa8YfNbeiWXtKCzsqiKsqzmVZELMVvclrN60Vw319fXnMwK7JhUhzHclXRuwGVZzmhZXClWo3Lxjw4oHQsSJfOHK7uM5aoeNo7MCiWdd09LeOlV_EuFJDF_GDEZm9rAld2hJ0_zJpeOfUQjDoVcj1w2EgX8z-QykPQX88_01WEMYYA9x_-I1V8NVi2fjyI30rdDPf7-P9gx1ePemn8jDw4acQlqjtBsl_S_AQAA__9AXTsF">