[llvm-dev] lld suspect behavior .group/rela/text input sections
Fāng-ruì Sòng via llvm-dev
llvm-dev at lists.llvm.org
Wed May 12 20:46:08 PDT 2021
On 2021-05-13, Moshtaghi, Alireza wrote:
>Hi
>The file contents and the commands to build and test are below. assuming that you have clang++, ld.lld, ld, sed, curl and gdb installed, the commands are at the end of this email.
>Just copy paste the lines below each file name into that file on your computer, then run the commands at the end. You’ll see that the file output linked with ld will load into gdb without any error but the output linked with ld.lld will produce errors when loaded into gdb
>
>[...]
The gdb command does not give me a warning/error. That said, I have simplified your example to the following:
cat > ./a.cc <<eof
inline int inline0() { return 0; }
inline int inline1() { return 1; }
int use() {
return inline0() + inline1();
}
eof
cat > ./a.sh <<eof
clang -c a.cc
ld.lld -r -T ldscript.amd64 a.o -o lld.ro
ld.bfd -r -T ldscript.amd64 a.o -o bfd.ro
readelf -g lld.ro
eof
cat > ./ldscript.amd64 <<eof
SECTIONS
{
.rela.text : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
.text1 : { *(.text .stub .text.* .gnu.linkonce.t.*) }
}
eof
zsh a.sh will give a readelf warning.
COMDAT group section [ 3] `.group' [_Z7inline0v] contains 1 sections:
[Index] Name
[ 2] .text1
COMDAT group section [ 4] `.group' [_Z7inline1v] contains 1 sections:
[Index] Name
readelf: Error: section [ 2] in group section [ 4] already in group section [ 3]
This is a case about what to do when the output would have two section groups
containing the same section. ELF specification says "A section cannot be a
member of more than one group." but relocatable output is not covered by the
specification.
It seems that GNU ld does not respect the *(... .text.*) input section
description for SHF_GROUP sections in this case. (It is unclear what exact
conditions it uses to decide whether to obey an input section description) It
simply doesn't place .text.* in .text, breaking the expectation of the input
section description usage. I agree with Peter that we are in the
implemention-defined realm. I currently don't see anything which should be
improved on LLD's side. My advice is to avoid such usage.
>-------------------- commands to build and load into gdb
>curl -o ldscript.amd64 https://raw.githubusercontent.com/freebsd/freebsd-src/master/sys/conf/ldscript.amd64
>sed -i 's/.*kern.*//g' ldscript.amd64
>sed -i 's/.*freebsd.*//g' ldscript.amd64
>clang++ -c file1.cpp -o file1.cpp.o
>clang++ -c file2.cpp -o file2.cpp.o
>clang++ -c file3.cpp -o file3.cpp.o
>ld.lld -r -T ldscript.amd64 *.o -o output.ld.lld
>ld -r -T ldscript.amd64 *.o -o output.ld.bfd
>echo "============== done building"
>gdb -batch -ex 'add-symbol-file output.ld.lld 0x1234'
>echo "============== done loading lld ( Notice the errors above)"
>gdb -batch -ex 'add-symbol-file output.ld.bfd 0x1234'
>echo "============== done loading lld (Notice that no error is produced here)"
>
>
>
>
>From: Fāng-ruì Sòng <maskray at google.com>
>Date: Monday, May 10, 2021 at 4:01 PM
>To: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com>
>Cc: llvm-dev at lists.llvm.org <llvm-dev at lists.llvm.org>, Peter Smith <Peter.Smith at arm.com>
>Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input sections
>NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
>On Mon, May 10, 2021 at 3:57 PM Moshtaghi, Alireza
><Alireza.Moshtaghi at netapp.com> wrote:
>>
>> sending it again to the list
>
>I am still waiting for a proper reproduce. Your commands have
>file1.cpp but your attachment has just file1.h.
>
>>
>>
>> From: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com>
>> Date: Saturday, May 1, 2021 at 10:53 PM
>> To: Fāng-ruì Sòng <maskray at google.com>
>> Cc: Peter Smith <Peter.Smith at arm.com>
>> Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input sections
>>
>> Oh
>>
>> Im sorry, I had typo in the file names
>>
>> file.h is the only header file; the other 3 are c++ files. (C may also do but I just use clang++)
>>
>> And of course you only compile the 3 files: file[1,2,3].cpp
>>
>> clang++ -c file1.cpp -o file1.cpp.o
>>
>> clang++ -c file2.cpp -o file2.cpp.o
>>
>> clang++ -c file3.cpp -o file3.cpp.o
>>
>> ld.lld -r -T ldscript.amd64 *.o -o output.ld.lld
>>
>> ld -r -T ldscript.amd64 *.o -o output.ld.bfd
>>
>> gdb -batch ex “add-symbol-file output.ld.lld 0x1234”
>> gdb -batch ex “add-symbol-file output.ld.bfd 0x1234”
>>
>>
>> ________________________________
>>
>> From: Fāng-ruì Sòng <maskray at google.com>
>> Sent: Friday, April 30, 2021 2:54:58 PM
>> To: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com>
>> Cc: Peter Smith <Peter.Smith at arm.com>
>> Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input sections
>>
>>
>>
>> NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>
>>
>>
>>
>> On 2021-04-30, Moshtaghi, Alireza wrote:
>> >Yes, the objects with COMDAT are good examples to show the problem.
>> >The problem with lld is when we supply a linker script. Without linker script, lld does the right thing.
>> >
>> >But with a linker script, it only follows the command to combine .text.* but does not combine respective .rela.text.*
>> >
>> >This effectively leaves loose ends in the output and makes them problematic with gdb
>> >
>> >Below there are 4 files. compile them with clang++ and then link with ld.lld as well as ld.bfd and use -r and -T to pass the linker script from the link from my first email in this thread. Call the outputs output.ld.lld and output.ld.bfd respectively.
>> >Then load the images into gdb like:
>> >gdb -batch ex “add-symbol-file output.ld.lld 0x1234”
>> >gdb -batch ex “add-symbol-file output.ld.bfd 0x1234”
>> >
>> >you’ll see that the second gdb command succeeds but first one fails and when I investigated I find out that the problem is that lld has merged the .text. _Z22second_inline_functionv and .text. _Z21first_inline_functionv into .text section but the associated .rela.text.* sections still exist in the output file and gdb fails to recognize these .rela sections because they are part of a group that is missing the text sections.
>> >
>> >ld.bfd leaves the .text.* sections alone and does not merge them so the group remains intact and gdb succeeds to load them.
>> >
>> >If I remove the .text.* from the linkerscript, ld.lld does the right thing but I don’t want to do this because that way the text sections that are not part of any group will also not being merged and that is not good.
>> >
>> >******** Minimal example :
>> >--------- file.h
>> >const char * get_char_value (void);
>> >const int * get_int_value (void);
>> >inline const char * first_inline_function (void) {
>> > return get_char_value ();
>> >}
>> >inline const int * second_inline_function (void) {
>> > return get_int_value ();
>> >}
>> >-------------------- file1.h
>> >#include "file.h"
>> >static int data = 5;
>> >const int *get_int_value (void) {
>> > return &data;
>> >}
>> >-------------------- file2.h
>> >#include "file.h"
>> >static const char *data = "is this really a meaningful?";
>> >const char *get_char_value (void) {
>> > return data;
>> >}
>> >-------------------- file3.h
>> >#include "file.h"
>> >const char * get_it (void) {
>> > const char *c = first_inline_function();
>> > const int *i = second_inline_function();
>> > return &c[*i];
>> >}
>>
>> I'm still puzzled. Do you miss some files?
>> clang -c file2.h produces a .gch file, unless you specify -xc++.
>>
>> If helps, I use the following script to print reproduce instructions.
>> Then users don't need to spend time creating files by themselves.
>>
>> #!/bin/zsh
>> dry_run=
>> if [[ $1 == --dry-run ]]; then dry_run=1; shift; fi
>>
>> f() {
>> for f in $1/*(ND.^x); do
>> [[ ${f##*/} =~ '^\.' ]] && continue
>> if [[ -n $dry_run ]]; then
>> echo $f
>> elif [[ $(wc -l < $f) == 1 ]]; then
>> echo "echo '$(<$f)' > $f"
>> else
>> echo "cat > $f <<eof"
>> cat $f
>> echo eof
>> fi
>> done
>> for d in $1/*(ND/); do
>> [[ $d =~ '^\.' ]] && continue
>> echo mkdir -p $d
>> f $d
>> done
>> }
>>
>> f ${1:-.}
>>
>> >
>> >From: Peter Smith <Peter.Smith at arm.com>
>> >Date: Thursday, April 29, 2021 at 3:38 AM
>> >To: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com>, Fāng-ruì Sòng <maskray at google.com>
>> >Subject: RE: [llvm-dev] lld suspect behavior .group/rela/text input sections
>> >NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> >
>> >
>> >I can reproduce part of it using ld -r on an object with a COMDAT group.
>> >
>> >Given that ld -r produces a relocatable object that can be used as input to a subsequent link step retaining the group information and the sections referenced by it uncollated in the ELF file makes sense to me. If the sections were collated into a single output section and the COMDAT group information lost then there would be multiple definition errors if the relocatable object were combined with other relocatable objects that had COMDAT groups with the same signature.
>> >
>> >Non COMDAT groups would be possible to collate without affecting subsequent links. Are the groups COMDAT or non COMDAT in your example?
>> >
>> >My limited understanding is that kernel modules use relocatable objects as kind of dynamically linked object, so there wouldn’t be any recombination, but the static linker doesn’t know what the output of ld -r will be used for.
>> >
>> >I don’t think the ELF spec covers combination of relocatable objects to output a single relocatable object. It is pretty much implementation defined behaviour from the linker.
>> >
>> >I’m not sure what to suggest as a resolution. It may be that a relocatable link needs an additional “kernel module” or “relocatable object” choice to guide the linker as to whether certain types of section should be combined.
>> >
>> >Peter
>> >
>> >From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Moshtaghi, Alireza via llvm-dev
>> >Sent: 28 April 2021 23:20
>> >To: Fāng-ruì Sòng <maskray at google.com>
>> >Cc: llvm-dev at lists.llvm.org
>> >Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input sections
>> >
>> >It is not easy to produce an minimal example. But I will try
>> >I was hoping that you may have solved it before.
>> >I will come back with an example
>> >
>> >Thanks
>> >A
>> >
>> >From: Fāng-ruì Sòng <maskray at google.com<mailto:maskray at google.com>>
>> >Date: Wednesday, April 28, 2021 at 3:11 PM
>> >To: Moshtaghi, Alireza <Alireza.Moshtaghi at netapp.com<mailto:Alireza.Moshtaghi at netapp.com>>
>> >Cc: llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
>> >Subject: Re: [llvm-dev] lld suspect behavior .group/rela/text input sections
>> >NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>> >
>> >
>> >
>> >
>> >On Wed, Apr 28, 2021 at 11:02 AM Moshtaghi, Alireza via llvm-dev
>> ><llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
>> >>
>> >> Hi
>> >>
>> >> Can you help me to understand if I’m doing anything wrong or the problem is with lld?
>> >>
>> >> I need to use linker-script (below link) for linking freebsd kernel modules and pass -r to ld.lld (version 10) to make these modules relocatable.
>> >>
>> >>
>> >>
>> >> Some of the modules that I build are very big and also have .group sections on rela.text.<mangledname> and .text.<mangledname>
>> >>
>> >>
>> >>
>> >> Per ELF specification these sections should be treated as one and either all merged or none. But I think lld is not respecting that spec and merges all .text.<mangledname> sections while leaving the .group and .rela.text.<mangledname> sections in the output, which results in broken elf (when I try to load it into gdb, I get Bad Value error)
>> >>
>> >>
>> >>
>> >> Alternatively, when I link with ld.bfd, the .text.<manglenames> remain in the output and I’m able to load the object into gdb without error.
>> >
>> >Hi, do you have a minimal reproducible example
>> >(https://stackoverflow.com/help/minimal-reproducible-example)?
>> >
>> >Freebsd is not build-friendly when the user is not using FreeBSD ;-)
>> >
>> >>
>> >>
>> >> Linker-script:
>> >>
>> >> https://github.com/freebsd/freebsd-src/blob/098dbd7ff7f3da9dda03802cdb2d8755f816eada/sys/conf/ldscript.amd64
>> >IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
>
>
>
>--
>宋方睿
More information about the llvm-dev
mailing list