[llvm-dev] Linking the FreeBSD base system with lld -- status update

[Rui Ueyama via llvm-dev]

Ed,

Thank you for the update! That is very informative, and it's very existing
to hear that LLD can now build a runnable FreeBSD system. It's a great
milestone. I personally want to build the entire FreeBSD system including
the kernel within six months or at least in this year, and I believe it is
a reasonable target based on the progress we've made so far in the last six
months to the ELF linker.

On Mon, Mar 7, 2016 at 2:24 PM, Ed Maste via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> As I've mentioned before[1] I've been regularly attempting to build
> the FreeBSD/amd64 base system with lld, in order to keep track of
> progress and identify issues on an ongoing basis. As of last November
> a 'buildword' (i.e., userland build) ran to completion, with several
> workarounds applied to the FreeBSD base system. However, the result
> did not actually work.
>
> I'm pleased to report that I can now build a runnable FreeBSD system
> using lld as the linker (for buildworld), with a few workarounds and
> work-in-progress patches. I have not yet extensively tested the result
> but it is possible to login to the resulting system, and basic sanity
> tests I've tried are successful. Note that the kernel is still linked
> with ld.bfd.
>
> I have tracking PR[2] 23214 open for lld issues affecting the FreeBSD
> base system use case, and I'll briefly summarize the outstanding
> issues. Unless otherwise specified my workaround is to use ld.bfd to
> link the affected program.
>
> 1. Symbol version support (PR 23231)
>
> FreeBSD relies on symbol version support in libc, libthr, and rtld for
> backwards compatibility and symbol visibility control.
>
> For testing I'm willing to build a system without symbol versioning.
> FreeBSD has a WITHOUT_SYMVER knob intended to enable this. It is
> currently broken, and I have patches[3][4] in review to fix it.
>
> I also added a "--hide-symbol" command to elfcopy, ELF Tool Chain's
> objcopy-equivalent. I use this to change the visibility of all symbols
> in libc_pic.a to hidden so that they do not end up incorrectly
> exported by ld-elf.so.1.
>
> 2. Linker script expression support (PR 26731)
>
> The FreeBSD kernel linker scripts contain expressions not supported by
> lld - for example, ". = ALIGN(. != 0 ? 64 / 8 : 1);". I'm using ld.bfd
> to link the kernel for now.
>

Comprehensive linker script support is the last major missing piece. It is
not technically very difficult to do, but if you aim a high performance
implementation (we do), it is a lot of stuff to consider, so we need a good
design before start implementing. I have a few ideas to prototype, but that
may take some time.

3. Library search paths
>
> In FreeBSD /usr/lib/libc.so is a linker script that contains "GROUP (
> libc.so.7 libc_nonshared.a libssp_nonshared.a )". ld.bfd includes a
> built-in /lib search path and finds /lib/libc.so.7 there. lld relies
> only on the -L paths specified on the command line, and cannot locate
> libc.so.7. As a workaround I've changed /usr/lib/libc.so to include
> the full path.
>

"/lib" and other system-default directories are added to the search path by
the default linker script in GNU ld.

I don't know how gold works. Does gold have the same issue as LLD?

4. -N/--omagic option
>
> -N makes the text and data sections RW and does not page-align data.
> It is used by boot loader components.
>

It is probably better to update FreeBSD boot loader rather than adding this
option (which is probably for '80s Unix compatibles).

5. -dc option
>
> -dc assigns space to common symbols when producing relocatable output
> (-r). It is used by the /rescue build, which is a single binary
> assembled from a collection of individual tools (sh, ls, fsck, ...)
>

Why does it need -dc option?


> 6. -Y option
>
> -Y adds a path to the default library search path. It is used by the
> lib32 build, which provides i386 builds of the system libraries for
> compatibility with i386 applications.
>

This option seems like an alias to -L. Why are they still using -Y?


> 7. Use of -r to convert a binary file into an ELF object
>
> A tool for loading firmware into a wireless USB device includes a
> built-in copy of the firmware image, and the image is converted to an
> ELF file using ld -r.
>

After George's commit to support -r, does it work now?


> The first two issues above are significant; the others can be
> addressed relatively easily but have simple workarounds and aren't
> holding up further progress.
>
> Thanks to all who have contributed to LLD's impressive progress over
> the last three months or so!
>
> [1] http://lists.llvm.org/pipermail/llvm-dev/2015-November/092572.html
> [2] http://llvm.org/pr23214
> [3] https://reviews.freebsd.org/D5571
> [4] https://reviews.freebsd.org/D5572
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160307/27e538af/attachment.html>