[llvm-dev] Linking the FreeBSD base system with lld -- status update

[Ed Maste via llvm-dev]

As I've mentioned before[1] I've been regularly attempting to build
the FreeBSD/amd64 base system with lld, in order to keep track of
progress and identify issues on an ongoing basis. As of last November
a 'buildword' (i.e., userland build) ran to completion, with several
workarounds applied to the FreeBSD base system. However, the result
did not actually work.

I'm pleased to report that I can now build a runnable FreeBSD system
using lld as the linker (for buildworld), with a few workarounds and
work-in-progress patches. I have not yet extensively tested the result
but it is possible to login to the resulting system, and basic sanity
tests I've tried are successful. Note that the kernel is still linked
with ld.bfd.

I have tracking PR[2] 23214 open for lld issues affecting the FreeBSD
base system use case, and I'll briefly summarize the outstanding
issues. Unless otherwise specified my workaround is to use ld.bfd to
link the affected program.

1. Symbol version support (PR 23231)

FreeBSD relies on symbol version support in libc, libthr, and rtld for
backwards compatibility and symbol visibility control.

For testing I'm willing to build a system without symbol versioning.
FreeBSD has a WITHOUT_SYMVER knob intended to enable this. It is
currently broken, and I have patches[3][4] in review to fix it.

I also added a "--hide-symbol" command to elfcopy, ELF Tool Chain's
objcopy-equivalent. I use this to change the visibility of all symbols
in libc_pic.a to hidden so that they do not end up incorrectly
exported by ld-elf.so.1.

2. Linker script expression support (PR 26731)

The FreeBSD kernel linker scripts contain expressions not supported by
lld - for example, ". = ALIGN(. != 0 ? 64 / 8 : 1);". I'm using ld.bfd
to link the kernel for now.

3. Library search paths

In FreeBSD /usr/lib/libc.so is a linker script that contains "GROUP (
libc.so.7 libc_nonshared.a libssp_nonshared.a )". ld.bfd includes a
built-in /lib search path and finds /lib/libc.so.7 there. lld relies
only on the -L paths specified on the command line, and cannot locate
libc.so.7. As a workaround I've changed /usr/lib/libc.so to include
the full path.

4. -N/--omagic option

-N makes the text and data sections RW and does not page-align data.
It is used by boot loader components.

5. -dc option

-dc assigns space to common symbols when producing relocatable output
(-r). It is used by the /rescue build, which is a single binary
assembled from a collection of individual tools (sh, ls, fsck, ...)

6. -Y option

-Y adds a path to the default library search path. It is used by the
lib32 build, which provides i386 builds of the system libraries for
compatibility with i386 applications.

7. Use of -r to convert a binary file into an ELF object

A tool for loading firmware into a wireless USB device includes a
built-in copy of the firmware image, and the image is converted to an
ELF file using ld -r.

The first two issues above are significant; the others can be
addressed relatively easily but have simple workarounds and aren't
holding up further progress.

Thanks to all who have contributed to LLD's impressive progress over
the last three months or so!

[1] http://lists.llvm.org/pipermail/llvm-dev/2015-November/092572.html
[2] http://llvm.org/pr23214
[3] https://reviews.freebsd.org/D5571
[4] https://reviews.freebsd.org/D5572