[cfe-dev] --gcc-toolchain (GCC_INSTALL_PREFIX) and --prefix(-B)

[Manoj Gupta via cfe-dev]

In the case of Chrome OS, we currently are using all 3 options:
"-B", "--prefix" and "--gcc-toolchain". "--gcc-toolchain" is only used when
the clang binary is not inside /usr/bin e.g. installed as
/opt/llvm/bin/clang.

I think the reason is the layout of gcc and binutils for cross compilation.
Example for aarch64:

binutils installation: /usr/libexec/gcc/aarch64-cros-linux-gnu  -> This is
path for "--prefix" (--prefix='
/usr/libexec/gcc/aarch64-cros-linux-gnu/aarch64-cros-linux-gnu-') and "-B"
(-B'/usr/libexec/gcc/aarch64-cros-linux-gnu/')
gcc binaries:
/usr/x86_64-pc-linux-gnu/aarch64-cros-linux-gnu/gcc-bin/10.2.0/
gcc libraries: /usr/lib/gcc/aarch64-cros-linux-gnu/10.2.0/
libc headers + libs: /usr/aarch64-cros-linux-gnu/usr/{include,lib64},
/usr/aarch64-cros-linux-gnu/lib64
sysroot: /build/device : Also has libc headers + libraries when building
for the device, substitutes libc files from /usr/aarch64-cros-linux-gnu
(--sysroot=/build/device).

The final command line looks like:
/opt/llvm/bin/clang --prefix='
/usr/libexec/gcc/aarch64-cros-linux-gnu/aarch64-cros-linux-gnu-'
-B'/usr/libexec/gcc/aarch64-cros-linux-gnu
--gcc-toolchain='/usr' --sysroot=/build/device .

My understanding is without passing "--gcc-toolchain=/usr", clang (when not
installed in /usr/bin) is not able to find libgcc from
/usr/lib/gcc/aarch64-cros-linux-gnu/10.2.0/. i.e. -B/--prefix options were
insufficient for finding gcc library paths.

Thanks,
Manoj

On Thu, Mar 4, 2021 at 3:14 PM Fāng-ruì Sòng via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> On Thu, Mar 4, 2021 at 2:31 PM Hubert Tong
> <hubert.reinterpretcast at gmail.com> wrote:
> >
> > On Thu, Mar 4, 2021 at 1:54 PM Fāng-ruì Sòng <maskray at google.com> wrote:
> >>
> >> On 2021-03-04, Hubert Tong wrote:
> >> >On Thu, Mar 4, 2021 at 3:19 AM Fāng-ruì Sòng via cfe-dev <
> >> >cfe-dev at lists.llvm.org> wrote:
> >> >
> >> >> If --gcc-toolchain is specified, its value overrides the cmake
> variable
> >> >> GCC_INSTALL_PREFIX.
> >> >> When the value is non-empty: the value is appended to the --prefix
> list
> >> >> and is used to detect GCC installations.
> >> >> The GCC installation is used to provide include directories/library
> >> >> directories and some startup files (e.g. crtbegin).
> >> >>
> >> >> Problem 1.
> >> >>
> >> >> --prefix(-B) does more than --gcc-toolchain:
> >> >> clang::driver::Driver::GetProgramPath basically searches for
> >> >> $prefix/$triple-$file and $prefix$file,
> >> >> where $prefix is taken from the list of --prefix(-B).
> --gcc-toolchain does
> >> >> not participate in the search. <have attached a summary of the
> algorithm at
> >> >> the bottom>
> >> >> The result is that 'ld' and 'as' may come from the system (more
> precisely,
> >> >> sysroot):
> >> >>
> >> >> cd clang/test/Driver
> >> >> # Make sure Inputs/opensuse_42.2_aarch64_tree/usr/bin/ld exists.
> >> >> clang -target aarch64-suse-linux
> >> >> --gcc-toolchain=Inputs/opensuse_42.2_aarch64_tree/usr '-###'
> >> >> gcc-toolchain.cpp -v
> >> >>
> >> >>    # I have ld in my /usr/local/bin and it takes precedence over
> >> >> /usr/bin/ld
> >> >>  "/usr/local/bin/ld" "-EL" "--eh-frame-hdr" "-m" "aarch64linux"
> >> >> "-dynamic-linker" "/lib/ld-linux-aarch64.so.1" "-o" "a.out"
> >> >>
> "Inputs/opensuse_42.2_aarch64_tree/usr/lib64/gcc/aarch64-suse-linux/4.8/../../../../lib64/crt1.o"
> >> >>
> "Inputs/opensuse_42.2_aarch64_tree/usr/lib64/gcc/aarch64-suse-linux/4.8/../../../../lib64/crti.o"
> >> >>
> "Inputs/opensuse_42.2_aarch64_tree/usr/lib64/gcc/aarch64-suse-linux/4.8/crtbegin.o"
> >> >>
> "-LInputs/opensuse_42.2_aarch64_tree/usr/lib64/gcc/aarch64-suse-linux/4.8"
> >> >>
> "-LInputs/opensuse_42.2_aarch64_tree/usr/lib64/gcc/aarch64-suse-linux/4.8/../../../../lib64"
> >> >> "-L/lib/aarch64-linux-gnu" "-L/lib/../lib64"
> "-L/usr/lib/aarch64-linux-gnu"
> >> >> "-L/usr/lib/../lib64"
> >> >>
> "-LInputs/opensuse_42.2_aarch64_tree/usr/lib64/gcc/aarch64-suse-linux/4.8/../../.."
> >> >> "-L/tmp/RelA/bin/../lib" "-L/lib" "-L/usr/lib"
> >> >> "/tmp/gcc-toolchain-f87f08.o" "-lgcc" "--as-needed" "-lgcc_s"
> >> >> "--no-as-needed" "-lc" "-lgcc" "--as-needed" "-lgcc_s"
> "--no-as-needed"
> >> >>
> "Inputs/opensuse_42.2_aarch64_tree/usr/lib64/gcc/aarch64-suse-linux/4.8/crtend.o"
> >> >>
> "Inputs/opensuse_42.2_aarch64_tree/usr/lib64/gcc/aarch64-suse-linux/4.8/../../../../lib64/crtn.o"
> >> >>
> >> >> The -L and crt* files are indeed from
> >> >> Inputs/opensuse_42.2_aarch64_tree/usr, but ld (and as if
> >> >> -fno-integrated-as) is from the system.
> >> >> On many Linux distributions you can normally assume that the system
> ld and
> >> >> as only support the host architecture.
> >> >> This means --gcc-toolchain can only be used to specify a GCC
> installation
> >> >> with the same architecture.
> >> >>
> >> >... and specifying a GCC installation with the same architecture
> (e.g., to
> >> >use a newer libstdc++) is something that people do:
> >> >$ clang++ -fuse-ld=ld --gcc-toolchain=/opt/rh/devtoolset-7/root -xc -
> >> >-fno-integrated-as <<<'#include <stdio.h>'$'\n''int main(void) {
> >> >printf("Hello, world!\n"); }' && echo OKAY
> >> >OKAY
> >> >... but it does seem ld is coming from --gcc-toolchain for me anyway:
> >> >$ ./build/bootstrap/stage1/build/bin/clang++ -fuse-ld=ld
> >> >--gcc-toolchain=/opt/rh/devtoolset-7/root -xc - -fno-integrated-as
> >> ><<<'#include <stdio.h>'$'\n''int main(void) { printf("Hello,
> world!\n"); }'
> >> >-### 2>&1 | grep /ld
> >> >
> "/opt/rh/devtoolset-7/root/lib/gcc/ppc64le-redhat-linux/7/../../../../bin/ld"
> >> >"--hash-style=gnu" "--eh-frame-hdr" "-m" "elf64lppc" "-dynamic-linker"
> >> >"/lib64/ld64.so.2" "-o" "a.out" "/lib/../lib64/crt1.o"
> >> >"/lib/../lib64/crti.o"
> >> >"/opt/rh/devtoolset-7/root/lib/gcc/ppc64le-redhat-linux/7/crtbegin.o"
> >> >"-L/opt/rh/devtoolset-7/root/lib/gcc/ppc64le-redhat-linux/7"
> >>
> >"-L/opt/rh/devtoolset-7/root/lib/gcc/ppc64le-redhat-linux/7/../../../../lib64"
> >> >"-L/lib/powerpc64le-linux-gnu" "-L/lib/../lib64"
> >> >"-L/usr/lib/powerpc64le-linux-gnu" "-L/usr/lib/../lib64"
> >> >"-L/opt/rh/devtoolset-7/root/lib/gcc/ppc64le-redhat-linux/7/../../.."
> >> >"-L/data1/hstong.local/wybuild/build/bootstrap/stage1/build/bin/../lib"
> >> >"-L/lib" "-L/usr/lib" "/tmp/--8f8c6a.o" "-lstdc++" "-lm" "-lgcc_s"
> "-lgcc"
> >> >"-lc" "-lgcc_s" "-lgcc"
> >> >"/opt/rh/devtoolset-7/root/lib/gcc/ppc64le-redhat-linux/7/crtend.o"
> >> >"/lib/../lib64/crtn.o"
> >>
> >> 'ld' path is computed by clang/lib/Driver/Driver.cpp
> Driver::GetProgramPath.
> >> Did you set PATH to the devtoolset-7 bin directory?
> >
> > No, I did not.
>
> Do you mind debugging clang '-###' a bit: set a breakpoint on
> clang/lib/Driver/Driver.cpp `GetProgramPath` and report how the 'ld'
> under --gcc-toolchain is selected?
>
> >>
> >>
> >> If --gcc-toolchain and multiple -B are specified, the search order is:
> >>
> >> -B -B -B --gcc-toolchain
> >>
> >> Currently the GCC installation with the largest version wins.
> >> Do you think we should stop search after one -B provides a GCC
> installation?
> >
> > Well, the ordering where --gcc-toolchain follows all of the -B seems
> okay in the context of looking for ld or as. For changing "largest version
> wins", it might make some sense to order --gcc-toolchain first for the
> resolution of the GCC installation (which allows "largest version wins" to
> continue with -B). I suppose it means people using builds preconfigured
> with a GCC install prefix will be somewhat inconvenienced when they don't
> want to use the preconfigured path; however, I think this retains most
> functionality (there is a way to limit the search for GCC installations but
> also a way to compose search paths for the latest GCC version).
>
> If cmake GCC_INSTALL_PREFIX was specified configuring llvm-project, I
> think it makes sense that explicit options (-B, --gcc-toolchain) take
> precedence over GCC_INSTALL_PREFIX.
> When --gcc-toolchain is specified: yes, probably it should take
> precedence over --prefix. I wonder whether it should suppress --prefix
> for GCC installation.
>
> I played with gcc -B a bit: looks like it picks the first --prefix
> where files can be found. So I am going to send a patch making the
> "largest version wins" local to one -B (i.e. if multiple versions are
> detected, pick the largest).
>
>
> >>
> >>
> >> >> --prefix can make as and ld paths correct, but: if another --prefix
> is
> >> >> needed, why do we use --gcc-toolchain?
> >> >>
> >> >Well, I'm not sure that another --prefix is needed. Also, --prefix
> doesn't
> >> >help to pick up the GCC header and library paths and --sysroot
> overrides
> >> >too much:
> >> >$ clang++ -fuse-ld=ld --sysroot=/opt/rh/devtoolset-7/root -xc -
> >> >-fno-integrated-as <<<'#include <stdio.h>'$'\n''int main(void) {
> >> >printf("Hello, world!\n"); }'
> >> ><stdin>:1:10: fatal error: 'stdio.h' file not found
> >> >#include <stdio.h>
> >> >         ^~~~~~~~~
> >> >1 error generated.
> >>
> >> (The example used --sysroot instead of --prefix)
> >
> > Yes.
> >
> >>
> >>
> >> OK, your other reply clarified that --prefix does pick up
> include/library directories.
> >
> > Yes.
>
>
>
> --
> 宋方睿
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20210304/ee07429a/attachment-0001.html>