[cfe-dev] [RFC] Recognize -fuse-ld={bfd, gold, lld} but don't prepend "ld." or "ld64." for other values

[Fangrui Song via cfe-dev]

On 2020-06-01, David Blaikie wrote:
>On Thu, May 28, 2020 at 7:12 PM Fangrui Song <maskray at google.com> wrote:
>>
>> On 2020-05-28, David Blaikie wrote:
>> >Adding the folks from the code review
>> >
>> >On Wed, May 20, 2020 at 12:42 PM Fangrui Song <maskray at google.com> wrote:
>> >>
>> >> On 2020-05-20, David Blaikie wrote:
>> >> >On Wed, May 20, 2020 at 12:10 PM Fangrui Song via cfe-dev <
>> >> >cfe-dev at lists.llvm.org> wrote:
>> >> >
>> >> >> In https://reviews.llvm.org/D80225 I intended to update the semantics of
>> >> >> -fuse-ld=
>> >> >> Some context (from my archaeology) about the option
>> >> >>
>> >> >> * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55470 added support for
>> >> >> -fuse-ld=bfd to mean ld.bfd and -fuse-ld=gold to mean ld.gold
>> >> >> * rL194328 ported the feature and made -fuse-ld= available for other
>> >> >> values (with the ld. prefix)
>> >> >> * D78290 changed the prefix to ld64. on Darwin.
>> >> >> * GCC added support for -fuse-ld=lld. No other value than bfd,gold,lld is
>> >> >> allowed (so our
>> >> >>    behavior change here will not be incompatible with GCC).
>> >> >>
>> >> >> Our existing behavior is cumbersome:
>> >> >>
>> >> >> * Inconsistency between an absolute
>> >> >>    path (which does not get a ld.  prefix) and a relative path (which gets
>> >> >>    a ld. prefix) For a relative path, -fuse-ld=dir/foo currently tries to
>> >> >>    access x86_64-unknown-linux-gnu-ld.dir/foo (if
>> >> >>    LLVM_DEFAULT_TARGET_TRIPLE is x86_64-unknown-linux-gnu).
>> >> >> * lld's new Mach-O port needs to pick an executable name. It would be
>> >> >>    nice if -fuse-ld= simply accepts a program name, not necessarily
>> >> >> prefixed by `ld64.`
>> >> >>
>> >> >
>> >> >To clarify (it seems like the link between "Mach-O lld needs to pick an
>> >> >executable name" and "it would be nice if -fuse-ld accepts a program name"
>> >> >isn't quite spelled out): By making this change to -fuse-ld it would enable
>> >> >the Mach-O version of lld to be called "lld" rather than "ld64.lld"? Is
>> >> >that the goal?
>> >>
>> >> The Mach-O port can be named whatever developers like, no need to
>> >> coupled with a `ld64.` prefix.
>> >
>> >Given the needs of a multiplatform/busyboxed linker like lld, it seems
>> >likely that any future multiplatform linker would probably use this
>> >same sort of prefix going forward.
>> >> I do think of whether -fuse-ld=lld means different things would be a
>> >> nice property: ld.lld (ELF), wasm-ld (wasm), lld-link (COFF) in
>> >> different contexts. I realized it is not. The linker options on
>> >> different binary formats are very different. Targeting different binary
>> >> formats requires one to customize the linker options anyway, a common
>> >> -fuse-ld=lld does not save much.
>> >
>> >Yep, doesn't seem likely - is the intent to use "-fuse-ld" on non-ELF
>> >platforms and make it easier/possible to write "-fuse-ld=lld-link"
>> >instead of having to provide the full path? (or does that already
>> >work/does using a non-ELF platform already disable the "ld."
>> >prefixing?)
>> >
>> >> (In 2010, it might be cleaner if binutils did not install ld.bfd -> ld
>> >> and ld.gold -> gold ...... They could juse use -fuse-ld=ld to mean "ld"
>> >> and -fuse-ld=gold to mean "gold")
>> >
>> >Sure enough - though dealing with where we are today is trickier :/
>> >
>> >Breaking existing users that depend on this generalized prefixing
>> >seems significant - Sony's the best known/most likely to hit this that
>> >I know of. Hopefully Paul Robinson can speak to that.
>>
>> clang::driver::tools::PS4cpu has its own logic
>> https://github.com/llvm/llvm-project/blob/master/clang/lib/Driver/ToolChains/PS4CPU.cpp#L341
>> It is not affected, but hope Paul can confirm.
>>
>> >If GCC is willing to make the same change, I don't feel too strongly
>> >about this - beyond the existing user breakage (which isn't an
>> >insignificant concern - breaking flag compatibility is something best
>> >avoided) the only future problem I could see (& given we're only
>> >currently living with 3 linkers, this isn't a huge deal) is that if
>> >another linker comes along there will be a subtle difference between
>> >"-fuse-ld=lld" and "-fuse-ld=newlink", at least because a user might
>> >be confused about exactly which binary is being used, in the former
>> >case it's "ld.lld" and in the latter it's "newlink". On top of that if
>> >that if "newlink" busyboxes, and perhaps the original name "newlink"
>> >either defaults to one platform (say MachO mode, like ld64.lld), but
>> >if you want to link for MachO, and busyboxes "ld.newlink" to provide
>> >the ld-compatible version - now you have to write
>> >"-fuse-ld=ld.newlink", but you don't have to do that with gold/etc...
>>
>> GCC does not support -fuse-ld=relative/path or -fuse-ld=/abs/path or
>> -fuse-ld=word_not_bfd_gold_lld
>>
>> I do have a patch to make it have a D80225 compatible behavior
>> https://gcc.gnu.org/pipermail/gcc-patches/2020-May/546277.html
>>
>> >Eh, such is life, perhaps.
>> >
>> >If the desire is just to support relative paths, that could be
>> >implemented by trying the prefix version - and if that fails, then
>> >trying it as a relative path? Though I can see value in sort of
>> >deprecating/baking in the prefix support only for the existing users
>> >and forcing all new usage to use the actual executable name.
>> >
>> >- Dave
>>
>> A relative path is supported.
>
>Oh, I thought when you said "For a relative path, -fuse-ld=dir/foo
>currently tries to access x86_64-unknown-linux-gnu-ld.dir/foo (if
>LLVM_DEFAULT_TARGET_TRIPLE is x86_64-unknown-linux-gnu)."
>
>You meant that was the binary it would run and fail - but you don't
>mean that, you mean that clang checks if that binary exists, then if
>it doesn't, it falls back to using "dir/foo" without a prefix? So it
>does what the user expected?

clang checks if x86_64-unknown-linux-gnu-ld.${fuse_ld} exists.  With
this patch, clang still checks x86_64-unknown-linux-gnu-${fuse_ld} (we
just removed an arbitrary prefix).  We shall think whether this behavior
makes sense and remove it if it doesn't. This can be done as a
follow-up.

>> The desire is to eliminate differences
>> between "filename" and "relative/filename". The platform specific prefix
>> rule (well, sometimes suffix, e.g. lld-link) is also hard to remember.
>
>Given you don't /have/ to remember it, that doesn't seem like a
>significant cost to users?
>
>> "trying it as a relative path" can gratuitously add more `access`
>> syscalls.  I initially tried this idea for my GCC patch. Martin Liška pointed out
>> the problem.
>>
>> (Note, we still have such a problem
>> access("/tmp/Debug/bin/x86_64-unknown-linux-gnu-ld.dir/foo", R_OK|X_OK) = -1 ENOENT (No such file or directory)
>> target triple prefixes may not make lots of sense for clang tools
>> )
>
>I don't think the extra 'stat' call is particularly
>significant/important to avoid - is there reason to believe it's
>especially costly or problematic compared to the totality of
>compiling/linking a program?
>
>So I'm now left wondering what problem is being fixed here - it sounds
>like the current LLVM implementation provides the prefix support where
>it works, and otherwise provides all the other support. The change
>you're proposing is to remove prefix support from all but the 3 common
>unix linkers - to remove one extra stat call when linking? Does that
>sum up the change?

Advantages:

* To drop an arbitrary prefix/suffix on some platforms.
   For lld in particular, we can pick a better name if people don't like
   the "ld64." prefix
* To make the rule more consistent. -fuse-ld={bfd,gold,lld} are
   hard-coded (unfortunate historical behavior), otherwise consistent.

>>
>>
>> >>
>> >> >Is Mach-O lld likely to need different command line compatibility than ELF
>> >> >or COFF lld? It seems like having a distinct name to enable detection of
>> >> >what command line support/object file format assumptions/other behavior
>> >> >changes compared to ELF/COFF may be useful, and "ld64.lld" is probably as
>> >> >good a name as any, if that functionality is desirable?
>> >> >
>> >> >>
>> >> >> I suggest we just hard code the currently used values which intend to get
>> >> >> a prefix: bfd, gold, lld. For all other values, don't add a prefix.
>> >> >> (PS4 seems to use -fuse-ld=ps4 but it overrides ToolChain::GetLinkerPath,
>> >> >> thus unaffected)
>> >> >>
>> >> >> Thoughts?
>> >> >> _______________________________________________
>> >> >> cfe-dev mailing list
>> >> >> cfe-dev at lists.llvm.org
>> >> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>> >> >>