[llvm-dev] LLD support for ld64 mach-o linker synthesised symbols
Michael Clark via llvm-dev
llvm-dev at lists.llvm.org
Wed Jun 7 15:39:45 PDT 2017
> On 8 Jun 2017, at 9:46 AM, Michael Clark <michaeljclark at mac.com> wrote:
>
>
>> On 8 Jun 2017, at 4:53 AM, Rui Ueyama <ruiu at google.com <mailto:ruiu at google.com>> wrote:
>>
>> On Tue, Jun 6, 2017 at 11:14 PM, Michael Clark via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> OK. I see that the Mach-O linker is not even built when LLD is enabled in Release_40, only the PE/COFF and ELF linkers are built.
>>
>> From looking at reviews it appears that Clang was able to be linked with LLD on Darwin about 2 years ago, so Mach-O support seems to have regressed.
>>
>> Only a few changes have been made to the Mach-O port in the last two years, so I'd doubt if it has regressed. It could be the case that clang's output has changed in such a way that the linker is not able to handle it.
>
> That’s actually good news!
>
> If there is a Mach-O linker that is able to self host Clang builds on macOS, then this is a really good starting point.
>
> From reading a tiny bit about the history, and the LLVM pages on the design of the various linkers, it seems like there is a difference in opinion with respect to the Atom based design of the Mach-O LLD, and whether or not there was to be an abstract design that supports ELF, PE/COFF and Mach-O. It seems not. One would also assume that LTO and/or -ffunction-sections -fdata-sections would obviate the need for Atoms, and that it may in fact increase the complexity of the linker.
>
> From my cursory examination of the source it seems that lld/lib should perhaps be renamed lld/MachO and become the MachO linker besides the ELF and COFF directproes as the common code is not being used by the ELF and the PE/COFF linkers.
besides the ELF and COFF directories (eyesight and spell checker fail).
> I just need to figure out how to build and invoke the Mach-O linker. There is no ‘ld’ in the llvm bin directory as one would be led to believe. I’ll dig into the CMakeLists.txt. I guess lld/lib//Driver/DarwinLdDriver.cpp is the entry point. lld//lib/Driver/CMakeLists.txt however only appears to define a library, versus an executable and there is no top level MachO directory like there is for the other 2 linkers.
>
> $ lld
> lld is a generic driver.
> Invoke ld.lld (Unix), ld (Mac) or lld-link (Windows) instead.
>
> $ ld.lld --version
> LLD 4.0.0
>
> $ lld-link --version
> ignoring unknown argument: --version
> error: no input files
I read the source. It seems the ‘ld’ symlink for the Mach-O linker is not being created for some reason. The ld.lld and llld-link symlinks are created correctly for the ELF and PE/COFF linkers.
In any case, I have the Mach-O linker invoking now. I will do some testing…
$ ln -s lld ld
$ ./ld
OVERVIEW: LLVM Linker
USAGE: ./ld [options] <inputs>
BUNDLE EXECUTABLE OPTIONS:
-bundle_loader <path> The executable that will be loading this Mach-O bundle
DYLIB EXECUTABLE OPTIONS:
-compatibility_version <version>
The dylib's compatibility version
-current_version <version>
The dylib's current version
-install_name <path> The dylib's install name
-mark_dead_strippable_dylib
Marks the dylib as having no side effects during initialization
LIBRARY OPTIONS:
-all_load Forces all members of all static libraries to be loaded
-force_load <library-path>
Forces all members of specified static libraries to be loaded
-F <dir> Add directory to framework search path
-L <dir> Add directory to library search path
-syslibroot <dir> Add path to SDK to all absolute library search paths
MAIN EXECUTABLE OPTIONS:
-export_dynamic Preserves all global symbols in main executables during LTO
-e <entry-name> entry symbol name
-no_pie Do not create Position Independent Executable
-pie Create Position Independent Executable (for ASLR)
-stack_size <value> Specifies the maximum stack size for the main thread in a program. Must be a page-size multiple. (default=8Mb)
OBSOLETE OPTIONS:
-multi_module Unsupported way to build dylibs
-objc_gc_compaction Unsupported ObjC GC option
-objc_gc_only Unsupported ObjC GC option
-objc_gc Unsupported ObjC GC option
-single_module Default for dylibs
OPTIMIZATIONS:
-data_in_code_info Force generation of a data in code load command
-dead_strip Remove unreference code and data
-exported_symbols_list <file-path>
Restricts which symbols will be exported
-exported_symbol <symbol>
Restricts which symbols will be exported
-flat_namespace Resolves symbols in any (transitively) linked dynamic libraries. Source libraries are not recorded: dyld will re-search all images at runtime and use the first definition found.
-function_starts Force generation of a function starts load command
-ios_simulator_version_min <version>
Minimum iOS simulator version
-ios_version_min <version>
Minimum iOS version
-keep_private_externs Private extern (hidden) symbols should not be transformed into local symbols
-macosx_version_min <version>
Minimum Mac OS X version
-mllvm <option> Options to pass to LLVM during LTO
-no_data_in_code_info Disable generation of a data in code load command
-no_function_starts Disable generation of a function starts load command
-no_objc_category_merging
Disables the optimisation which merges Objective-C categories on a class in to the class itself.
-no_version_load_command
Disable generation of a version load command
-order_file <file-path> re-order and move specified symbols to start of their section
-sdk_version <version> SDK version
-source_version <version>
Source version
-twolevel_namespace Resolves symbols in listed libraries only. Source libraries are recorded in the symbol table.
-undefined <undefined> Determines how undefined symbols are handled.
-unexported_symbols_list <file-path>
Lists symbols that should not be exported
-unexported_symbol <symbol>
A symbol which should not be exported
-version_load_command Force generation of a version load command
OPTIONS:
-arch <arch-name> Architecture to link
-demangle Demangles symbol names in errors and warnings
-dependency_info <file> Write binary list of files used during link
-filelist <path> file containing paths to input files
-framework <name> Base name of framework searched for in -F directories
-l<libname> Base name of library searched for in -L directories
-o <path> Output file path
-path_exists <path> Used with -test_file_usage to declare a path
-print_atoms Emit output as yaml atoms
-rpath <path> Add path to the runpath search path list for image being created
-sectalign <segname> <sectname> <alignment>
Alignment for segment/section
-sectcreate <segname> <sectname> <file>
Create section <segname>/<sectname> from contents of <file>
-S Remove debug information (STABS or DWARF) from the output file
-test_file_usage Only files specified by -file_exists are considered to exist. Print which files would be used
-t Print the names of the input files as ld processes them
-upward-l<libname> Base name of upward library searched for in -L directories
-upward_framework <name>
Base name of upward framework searched for in -F directories
-upward_library <path> path to upward dylib to link with
-v Print linker information
-Z Do not search standard directories for libraries or frameworks
OUTPUT KIND:
-bundle Create dynamic bundle
-dylib Create dynamic library
-dynamic Create dynamic executable (default)
-execute Create main executable (default)
-preload Create binary for use with embedded systems
-r Create relocatable object file
-static Create static executable
> If I know which CMakeLists.txt defines the binary that hosts the main function and installs it, then I can take it from there.
>
>> Curious as to pointers to primordial branches with whatever needs to be resurrected. I couldn’t find any Mach-O cmake flags to enable its build. A pointer to a branch or tag that might have a working Mach-O LLD would be a start.
>>
>>
>>> On 7 Jun 2017, at 11:38 AM, Michael Clark <michaeljclark at mac.com <mailto:michaeljclark at mac.com>> wrote:
>>>
>>> Hi Rui,
>>>
>>> The motivation would be primarily that LLVM/Clang/LLD are community projects such that if I or someone in the community added support for e.g. symbol aliases, then it could be reviewed and potentially merged. ld64 on the other hand does not have a community process for patch submission and code review that I am aware of so its unlikely that if someone from the community came up with a patch to support aliases that it would be merged.
>>>
>>> In that case I might check out the LLD code and try linking “x86_64-xnu-musl” with it. My requirements are likely simpler than Apple’s however I do need symbol aliases and these are not supported by ld64. The linker synthesised symbols are likely not too difficult to add if they are not present… now on my to do list…
>>>
>>> Michael.
>>>
>>>> On 7 Jun 2017, at 11:30 AM, Rui Ueyama <ruiu at google.com <mailto:ruiu at google.com>> wrote:
>>>>
>>>> Hi Michael,
>>>>
>>>> The Mach-O version of LLD is not being developed actively, and if some feature is missing, it is likely that it's just not implemented. What is your motivation to use LLD instead of ld64?
>>>>
>>>> On Tue, Jun 6, 2017 at 4:08 PM, Michael Clark via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>> Hi Folks,
>>>>
>>>> I have a question regarding LLD support for ld64 mach-o linker synthesised symbols. I did a quick search of the LLD source and I can not find support for them so before I start trying to use lld I thought I would ask.
>>>>
>>>> I have found a couple of cases where they are essential. i.e. where there is no other way to get the required information, such as getting the address of the mach-o headers of the current process, with ASLR enabled, if the process is not dyld as exec on macOS only provides the mach header address to dyld (*1). They are used inside of dyld and I am now using them in “x86_64-xnu-musl”.
>>>>
>>>> It’s possible to resolve a mach-o segment offset or a mach-o section offset using these special ld64 linker synthesised symbols. See resolveUndefines:
>>>>
>>>> - https://opensource.apple.com/source/ld64/ld64-274.2/src/ld/Resolver.cpp.auto.html <https://opensource.apple.com/source/ld64/ld64-274.2/src/ld/Resolver.cpp.auto.html>
>>>>
>>>> There are 4 special symbol prefixes for the mach-o linker synthesised symbols:
>>>>
>>>> - segment$start$__SEGMENT
>>>> - segment$end$__SEGMENT
>>>> - section$start$__SEGMENT$__section
>>>> - section$end$__SEGMENT$__section
>>>>
>>>> In asm:
>>>>
>>>> /* get imagebase and slide for static PIE and ASLR support in x86_64-xnu-musl */
>>>>
>>>> .align 3
>>>> __image_base:
>>>> .quad segment$start$__TEXT
>>>> __start_static:
>>>> .quad start
>>>> .text
>>>> .align 3
>>>> .global start
>>>> start:
>>>> xor %rbp,%rbp
>>>> mov %rsp,%rdi
>>>> andq $-16,%rsp
>>>> movq __image_base(%rip), %rsi
>>>> leaq start(%rip), %rdx
>>>> subq __start_static(%rip), %rdx
>>>> call __start_c
>>>>
>>>> In C:
>>>>
>>>> /* run C++ constructors in __libc_start_main for x86_64-xnu-musl */
>>>>
>>>> typedef void (*__init_fn)(int, char **, char **, char **);
>>>> extern __init_fn __init_start __asm("section$start$__DATA$__mod_init_func");
>>>> extern __init_fn __init_end __asm("section$end$__DATA$__mod_init_func”);
>>>>
>>>> static void __init_mod(int argc, char **argv, char **envp, char **applep)
>>>> {
>>>> for (__init_fn *p = &__init_start; p < &__init_end; ++p) {
>>>> (*p)(argc, argv, envp, applep);
>>>> }
>>>> }
>>>>
>>>> Michael.
>>>>
>>>> [1] https://github.com/opensource-apple/xnu/blob/dc0628e187c3148723505cf1f1d35bb948d3195b/bsd/kern/kern_exec.c#L1072-L1111 <https://github.com/opensource-apple/xnu/blob/dc0628e187c3148723505cf1f1d35bb948d3195b/bsd/kern/kern_exec.c#L1072-L1111>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170608/1961f448/attachment-0001.html>
More information about the llvm-dev
mailing list