[cfe-dev] Feedback on feature (and status of lld linker script support?)

[Fangrui Song via cfe-dev]

On 2021-11-29, David Blaikie wrote:
>+Fangrui Song <maskray at google.com> for general LLD stuff
>+Jez Ng <jezng at fb.com> for ld64.lld stuff
>+Reid Kleckner <rnk at google.com> for discussion around section/symbol
>reduction
>
>I'm guessing it'll be best to keep each issue in a separate thread - and
>the previous thread on llvm-dev (
>https://groups.google.com/g/llvm-dev/c/YzHbrfTxLCY/m/ssYtlxIdAwAJ )
>
>The section name hashing might be more of a clang feature than an LLVM one
>- if it's specifically for user-specified sections, etc. It could have some
>overlap with some symbol name length issues I've been considering for a
>while with somewhat similar strategies (hash, but then have a lookup
>table). I'm not sure who's going to be best to actually make the final call
>on whether what you're proposing is suitable for Clang/LLVM

(I use this subthread for the 16-byte section name length limitation.)

It would be nice to fix the 16-byte section name limitation for Mach-O.
For many instrumentations, for binary format portability, we have to restrict the section name length even
when the length size doesn't cost that much.
I think this probably should be fixed at the binary format level.
For example, PE/COFF had a 7-char limitation and extended it with
https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#section-table-section-headers
("For longer names, this field contains a slash (/) that is followed by
an ASCII representation of a decimal number that is an offset into the
string table.")

The rationale is that this is a limitation that the binary format maker
would likely have an opinion on and needs to change many things in the
ecosystem, e.g.

* does the assembler need to do anything special?
* how do otool/llvm-objdump/llvm-readobj render the section name?
* how do ld64/ld64.lld render the section name, especially in diagnostics
* whether the custom section can be stripped by binary manipulation tools
* how section$start$__SEG$__section symbols work with the section representation
* ...

If the section name length is a problem, the binary format maker can use
compression on the section name part (see __llvm_prf_nm).

>On Sun, Nov 28, 2021 at 7:53 AM Christopher Friedt via cfe-dev <
>cfe-dev at lists.llvm.org> wrote:
>
>> Hi all,
>>
>> I emailed the llvm mailing list but have not heard any response yet,
>> but maybe I was sent to the wrong list. Hopefully this is the right
>> one.
>>
>> TLDR:
>> 1) Is anyone available to provide some quick feedback on a feature [1]
>> before I submit it?
>> 2) What is the status of linker script support for LLD on macOS?
>>
>> I'm a bit of a newcomer to clang / llvm development and I'm working on
>> a feature that I hope will improve the experience on macOS for
>> applications that rely on custom section placement. Specifically, it
>> addresses this type of error:
>>
>> error: argument to 'section' attribute is not valid for this target:
>> mach-o section specifier requires a section whose length is between 1
>> and 16 characters
>>
>> The solution I've put together in [1] simply hashes long section names
>> to fit into N (16) characters. The approach is SHA256 -> base64 ->
>> truncate to N which is already somewhat common in various runtimes on
>> macOS.
>>
>> I've added two options:
>> * -fhash-section-names=N
>>   * hashes section names when they are > N characters
>> * -fhashed-section-names=outputfile
>>   * outputs a mapping of all section names to the hash value in tabbed
>> CSV format
>>   * each line is of the form: (section)\t(hash)
>>   * would be nice to reuse any artifact file names if "outputfile" is ""
>>   * e.g. build/foo.o => build/foo.sn, build/foo.exe => foo.sn,
>> build/foo => foo.sn
>>
>> The above changes allow one to use linker-generated symbols to iterate
>> over all elements placed into a particular section. It is working
>> quite well for local "native_posix_64" builds of Zephyr so far. Zephyr
>> uses (long) custom section names extensively. Otherwise the section
>> name size limitation of macOS is a showstopper.
>>
>> Does anyone have any quick feedback before I submit this patch to the
>> mailing list?
>>
>> The second question I had was about LLD support for GNU style linker
>> scripts on macOS. I'm currently getting this error:
>>
>> ld64.lld: error: zephyr/linker_zephyr_pre0.cmd: unhandled file type
>>
>> Is there an option I can provide to LLD that would provide more
>> information about the error? Currently I'm using "-Wl,-flat_namespace,
>> -Wl,-undefined,warning" just to make it link, but there are several
>> symbols defined in the linker script that are required for the
>> application to run properly. Supporting GNU style linker scripts would
>> be immensely helpful in this case, so I'd like to make that work if at
>> all possible. Would be happy to patch LLD as well.
>>
>> Thanks,
>>
>> C
>>
>> [1]
>> https://github.com/llvm/llvm-project/compare/main...cfriedt:fhash-long-section-names
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>