[lld] r238115 - [ELF] Fix lld when no unique sections is used
Sean Silva
chisophugis at gmail.com
Wed May 27 19:49:26 PDT 2015
On Wed, May 27, 2015 at 6:33 PM, Rafael Espíndola <
rafael.espindola at gmail.com> wrote:
> Joining late to thread, but here is my opinion on the binary file issue:
>
> Split the set of all possible inputs in two
>
> * Simple inputs. These are valid files with no requirements about some
> implementation detail. For example, a file with an undefined reference
> to "foo", but doesn't care what is the symbol number of foo.
> * Complex inputs. These include all invalid/corrupted objects but also
> cases that depend on specific implementation details, like the
> particular layout of strings in a .strtab.
>
> For the first case, one can just use llvm-mc. An assembly input is far
> easier to read and maintain than yaml.
>
> For the second case, I think checking in the binary is a superior
> option. It makes sure we are testing the actual feature of the linker
> that was broken before. Maintenance is not an issue. We want to keep
> lld working with files that have multiple sections with the same name
> no meter what. Even if we decide those files are a bad idea and nuke
> generation from llvm, they are valid ELF and we should keep supporting
> them. Given how specific these files are, understanding them is also
> not a problem. Taking an example from the bitcode reader, can anyone
> guess what is the property of invalid-global-var-comdat-id.bc? :-)
>
I agree that if a test can be written with llvm-mc, it probably makes sense
to write it with llvm-mc. I bet that quite a few LLD tests that currently
are using yaml could be switched over.
But there are relatively few easy ways to create object files that an
assembler/compiler wouldn't create. yaml2obj is probably the easiest and
least-error-prone for certain things. For example, to test error handling
of certain things related to relocations, you can't use an assembler, but
it is also requires creating object files that reference specific
relocations with specific values of fields. 010 editor with the ELF binary
template is the only tool that I know of that would let you moderately
conveniently create files like
lld/test/elf/AArch64/rel-abs16-overflow.test, whereas the yaml is pretty
straightforward and is just a simple textual edit from the "regular case"
test of the relocation, as you would expect for a test which is meant to
differ from the working case in just a single number.
So from a developer's perspective it retains the feeling of "ok, here's a
test for the usual case" and "ok, here's a test for the error case";
whereas the alternatives I can think of using llvm-mc + editing the binary
make testing the error case seem like a huge chore by comparison, and
likely the error case would just be untested.
So I think there's a gray area between your classification of "Simple
inputs" and "Complex inputs" where yaml2obj is actually a pretty decent fit.
-- Sean Silva
>
> Cheers,
> Rafael
>
>
> On 26 May 2015 at 19:26, Sean Silva <chisophugis at gmail.com> wrote:
> >
> >
> > On Tue, May 26, 2015 at 2:18 PM, Rui Ueyama <ruiu at google.com> wrote:
> >>
> >> On Tue, May 26, 2015 at 2:08 PM, David Blaikie <dblaikie at gmail.com>
> wrote:
> >>>
> >>>
> >>>
> >>> On Sun, May 24, 2015 at 9:19 AM, Simon Atanasyan <simon at atanasyan.com>
> >>> wrote:
> >>>>
> >>>> Author: atanasyan
> >>>> Date: Sun May 24 11:19:27 2015
> >>>> New Revision: 238115
> >>>>
> >>>> URL: http://llvm.org/viewvc/llvm-project?rev=238115&view=rev
> >>>> Log:
> >>>> [ELF] Fix lld when no unique sections is used
> >>>>
> >>>> Original patch of Shankar Easwaran with additional test case.
> >>>> The yaml2obj does not allow to create an object file with non-unique
> >>>> sections names
> >>>
> >>>
> >>> Should we fix yaml2obj?
> >>
> >>
> >> Yes we should fix it if it can't handle valid objects, but I think we
> >> probably start considering checking in object files (as opposed to
> checking
> >> in yaml files) is the right thing, as IIRC Rafael said before. Object
> files
> >> are the input for the linker, so in order to get a precise control over
> the
> >> input, it's better to use object files themselves.
> >
> >
> > Rafael was referring to using a binary as essentially an integration test
> > "to test that lld can work with a binary produced by a current clang".
> For
> > the bulk of testing that is definitely the wrong thing to do.
> >
> > The idea that we need "precise control" in general is patently false.
> >
> > For a given test input, we expect the same behavior on a class of
> equivalent
> > object files. What is considered equivalent is determined by what
> > transformations can be made to an object file that will result in the
> same
> > interpretation by the code being tested. For example, we can rearrange
> the
> > string table with no effect on the interpretation of the object file.
> >
> > When testing a particular piece of code by feeding an input, the only
> > important thing is that we provide an input that falls into a class of
> > equivalent object files that exercise the code in the way we want. It is
> in
> > that sense that the output of yaml2obj is useful for testing. It is the
> same
> > reason that the .ll format is useful for testing. The .ll format cannot
> > represent all .bc files, nor can it represent all in-memory Module's, but
> > that is a *good thing*, because for most of the things that we want to
> test,
> > it *is* able to be transformed into a member of the equivalence class of
> > in-memory Module's that we need for testing, while being substantially
> more
> > readable (and other good properties).
> >
> > -- Sean Silva
> >
> >>
> >>
> >>>
> >>>>
> >>>> so the fix uses a binary input object file in the test
> >>>> case.
> >>>>
> >>>> Added:
> >>>> lld/trunk/test/elf/Inputs/no-unique-section-names.x86-64
> >>>> lld/trunk/test/elf/no-unique-section-names.test
> >>>> Modified:
> >>>> lld/trunk/lib/ReaderWriter/ELF/ELFFile.cpp
> >>>> lld/trunk/lib/ReaderWriter/ELF/ELFFile.h
> >>>>
> >>>> Modified: lld/trunk/lib/ReaderWriter/ELF/ELFFile.cpp
> >>>> URL:
> >>>>
> http://llvm.org/viewvc/llvm-project/lld/trunk/lib/ReaderWriter/ELF/ELFFile.cpp?rev=238115&r1=238114&r2=238115&view=diff
> >>>>
> >>>>
> ==============================================================================
> >>>> --- lld/trunk/lib/ReaderWriter/ELF/ELFFile.cpp (original)
> >>>> +++ lld/trunk/lib/ReaderWriter/ELF/ELFFile.cpp Sun May 24 11:19:27
> 2015
> >>>> @@ -143,7 +143,7 @@ std::error_code ELFFile<ELFT>::createAto
> >>>> auto rai(_objFile->begin_rela(§ion));
> >>>> auto rae(_objFile->end_rela(§ion));
> >>>>
> >>>> - _relocationAddendReferences[*sectionName] = make_range(rai,
> rae);
> >>>> + _relocationAddendReferences[sHdr] = make_range(rai, rae);
> >>>> totalRelocs += std::distance(rai, rae);
> >>>> } else if (section.sh_type == llvm::ELF::SHT_REL) {
> >>>> auto sHdr = _objFile->getSection(section.sh_info);
> >>>> @@ -155,7 +155,7 @@ std::error_code ELFFile<ELFT>::createAto
> >>>> auto ri(_objFile->begin_rel(§ion));
> >>>> auto re(_objFile->end_rel(§ion));
> >>>>
> >>>> - _relocationReferences[*sectionName] = make_range(ri, re);
> >>>> + _relocationReferences[sHdr] = make_range(ri, re);
> >>>> totalRelocs += std::distance(ri, re);
> >>>> } else {
> >>>> _sectionSymbols[§ion];
> >>>> @@ -554,12 +554,12 @@ ELFDefinedAtom<ELFT> *ELFFile<ELFT>::cre
> >>>> unsigned int referenceStart = _references.size();
> >>>>
> >>>> // Add Rela (those with r_addend) references:
> >>>> - auto rari = _relocationAddendReferences.find(sectionName);
> >>>> + auto rari = _relocationAddendReferences.find(section);
> >>>> if (rari != _relocationAddendReferences.end())
> >>>> createRelocationReferences(symbol, symContent, rari->second);
> >>>>
> >>>> // Add Rel references.
> >>>> - auto rri = _relocationReferences.find(sectionName);
> >>>> + auto rri = _relocationReferences.find(section);
> >>>> if (rri != _relocationReferences.end())
> >>>> createRelocationReferences(symbol, symContent, secContent,
> >>>> rri->second);
> >>>>
> >>>>
> >>>> Modified: lld/trunk/lib/ReaderWriter/ELF/ELFFile.h
> >>>> URL:
> >>>>
> http://llvm.org/viewvc/llvm-project/lld/trunk/lib/ReaderWriter/ELF/ELFFile.h?rev=238115&r1=238114&r2=238115&view=diff
> >>>>
> >>>>
> ==============================================================================
> >>>> --- lld/trunk/lib/ReaderWriter/ELF/ELFFile.h (original)
> >>>> +++ lld/trunk/lib/ReaderWriter/ELF/ELFFile.h Sun May 24 11:19:27 2015
> >>>> @@ -345,10 +345,10 @@ protected:
> >>>> /// list of relocations references. In ELF, if a section named,
> >>>> ".text" has
> >>>> /// relocations will also have a section named ".rel.text" or
> >>>> ".rela.text"
> >>>> /// which will hold the entries.
> >>>> - std::unordered_map<StringRef, range<Elf_Rela_Iter>>
> >>>> + std::unordered_map<const Elf_Shdr *, range<Elf_Rela_Iter>>
> >>>> _relocationAddendReferences;
> >>>> MergedSectionMapT _mergedSectionMap;
> >>>> - std::unordered_map<StringRef, range<Elf_Rel_Iter>>
> >>>> _relocationReferences;
> >>>> + std::unordered_map<const Elf_Shdr *, range<Elf_Rel_Iter>>
> >>>> _relocationReferences;
> >>>> std::vector<ELFReference<ELFT> *> _references;
> >>>> llvm::DenseMap<const Elf_Sym *, Atom *> _symbolToAtomMapping;
> >>>> llvm::DenseMap<const ELFReference<ELFT> *, const Elf_Sym *>
> >>>>
> >>>> Added: lld/trunk/test/elf/Inputs/no-unique-section-names.x86-64
> >>>> URL:
> >>>>
> http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/Inputs/no-unique-section-names.x86-64?rev=238115&view=auto
> >>>>
> >>>>
> ==============================================================================
> >>>> Binary files lld/trunk/test/elf/Inputs/no-unique-section-names.x86-64
> >>>> (added) and lld/trunk/test/elf/Inputs/no-unique-section-names.x86-64
> Sun May
> >>>> 24 11:19:27 2015 differ
> >>>>
> >>>> Added: lld/trunk/test/elf/no-unique-section-names.test
> >>>> URL:
> >>>>
> http://llvm.org/viewvc/llvm-project/lld/trunk/test/elf/no-unique-section-names.test?rev=238115&view=auto
> >>>>
> >>>>
> ==============================================================================
> >>>> --- lld/trunk/test/elf/no-unique-section-names.test (added)
> >>>> +++ lld/trunk/test/elf/no-unique-section-names.test Sun May 24
> 11:19:27
> >>>> 2015
> >>>> @@ -0,0 +1,19 @@
> >>>> +# Check handling object files with non-unique named sections.
> >>>> +
> >>>> +RUN: lld -flavor gnu -target x86_64-linux -shared -o %t \
> >>>> +RUN: %p/Inputs/no-unique-section-names.x86-64
> >>>> +RUN: llvm-objdump -s %p/Inputs/no-unique-section-names.x86-64 %t \
> >>>> +RUN: | FileCheck %s
> >>>> +
> >>>> +CHECK: Contents of section .group:
> >>>> +CHECK-NEXT: 0000 01000000 08000000
> >>>> +CHECK-NEXT: Contents of section .text:
> >>>> +CHECK-NEXT: 0000 [[A1:[0-9a-f]+]] [[A2:[0-9a-f]+]] [[A3:[0-9a-f]+]]
> >>>> +CHECK-NEXT: Contents of section .group:
> >>>> +CHECK-NEXT: 0000 01000000 0a000000
> >>>> +CHECK-NEXT: Contents of section .text:
> >>>> +CHECK-NEXT: 0000 [[B1:[0-9a-f]+]] [[B2:[0-9a-f]+]] [[B3:[0-9a-f]+]]
> >>>> +
> >>>> +CHECK: Contents of section .text:
> >>>> +CHECK: {{[0-9a-f]+}} [[A1]] [[A2]] [[A3]]
> >>>> +CHECK-NEXT: {{[0-9a-f]+}} [[B1]] [[B2]] [[B3]]
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> llvm-commits mailing list
> >>>> llvm-commits at cs.uiuc.edu
> >>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
> >>>
> >>>
> >>
> >>
> >> _______________________________________________
> >> llvm-commits mailing list
> >> llvm-commits at cs.uiuc.edu
> >> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
> >>
> >
> >
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150527/db04ba0e/attachment.html>
More information about the llvm-commits
mailing list