[llvm-dev] [RFC] Adding support for dynamic entries in yaml2obj

Thu Jan 17 17:16:20 PST 2019

1) Producing an error in this situation is reasonable. Dynamic symbols
don’t produce an error when .dynstr has explicit content specified, and
that has been a source of confusion for me before.
2) For the sake of simplicity, strings won’t be added to the linked section
unless the linked section is the default .dynstr. In all other cases, a
numeric offset value must be used to specify the location of a string. I’ve
provided more details on this at the end of this reply.
3) I was initially considering this as well, and the original patch already
does this for a few tags. If that’s generally desirable, I’m game to keep
this in and improve control over automatic generation of tags. The current
behavior is that user defined tags have precedence over the auto-generated
tags.
4) This is already implemented in yaml2obj for all section types (though
the symbol tables forcibly override the linked section if symbols are
specified). As my patch stands right now, .dynamic currently links to
.dynstr by default, but only if dynamic entries have been specified. I
personally feel that is appropriate behavior. Manually specifying “Link: 0”
should properly override the default in the case that dynamic entries are
specified. Alternatively, if no dynamic entries are specified, the linked
section defaults to 0.

Pretty much every use case you’ve brought up is either already implemented
in my patch, or wouldn’t be too difficult to add. The only major exception
would be allowing strings (for tags like DT_SONAME) to be added to string
tables other than .dynstr. The existing design of yaml2obj doesn’t make
that very approachable. It’s more reasonable to attempt to search for the
specified string in the linked section, but that would require some
significant structural changes as well.

On Thu, Jan 17, 2019 at 1:55 AM James Henderson <
jh7370.2008 at my.bristol.ac.uk> wrote:

> Thanks for bringing this up. Since you posted on the review, I've been
> thinking more about the different options and overall design of yaml2obj's
> dynamic sections (including dynstr and dynsym) and how they work from a
> user's perspective, not least motivated by https://reviews.llvm.org/D56791,
> where I had to hand-craft a large .dynamic section, and ended up fighting
> with .dynsym and .dynstr too (see also
> https://bugs.llvm.org/show_bug.cgi?id=40339, for example).
>
> As for the proposal, it sounds reasonable to be able to specify numeric
> and string arguments as you propose. However, I do have some
> questions/thoughts/points:
>
> 1) What should happen if an explicit .dynstr content is specified (or more
> specifically, explicit content is specified for the linked section, see
> point 2)? My suggestion would be that it should be an error to specify
> string values in .dynamic and explicit content in .dynstr at the same time.
> 2) I'd like to avoid the same issue that is present in
> https://bugs.llvm.org/show_bug.cgi?id=40337, namely that the .dynamic
> section auto-populates strings in .dynstr, even though it is linked against
> some other section. Note that this other section could also be a string
> table, so using strings in that instance would still be valid.
> 3) It would be nice to have some mechanism to optionally auto-populate the
> .dynamic section with DT_STRTAB, DT_STRSZ, DT_SYMTAB, etc.
> 4) It should be possible to omit the "Link:" field of the header, to get a
> value of 0 there, or to specify a section index, in place of a section name.
>
> Related to point 3), I foresee several different use-cases: a) users want
> a regular .dynamic section, with DT_STRTAB etc in, but want the values
> auto-populated for everything. They don't want to specify any tags at all.
> yaml2obj does the hard work of fetching the addresses and sizes
> automatically; b) similar to a) but a user wants to be able to extend a
> .dynamic section with other tags (e.g. DT_SONAME); c) again similar to a),
> but a user wants to be able to override some of the auto-generated tags
> (but not necessarily all of them); d) a user wants to completely control
> the content with normal-looking tags, i.e. no auto-generation at all (e.g.
> they want to create a dynamic section without DT_STRTAB); e) a user wants
> to be able to hand-craft the content completely (e.g. to create truncated
> tags etc) - this could probably be best done via having an explicit
> content. I think all except e) could be accomplished by having an extra
> attribute for dynamic sections, namely something like "DeriveTags:
> true/false" or similar. A value of true would mean that all mandatory tags
> are automatically generated, unless an explicit tag of the same value is
> specified. In order to have multiple default-generated tags with the same
> DT_* value (or none of them), this would need to be false.
>
> I like your idea of a missing Value being inferred. This would work well
> with the ability to turn off auto-generated tags, and would minimise the
> pain of having to write each normally-defaulted tag by hand in this case.
>
> Note: I have no idea how well my above proposal works in relation to the
> current design of yaml2obj. My personal ideal is that a user can use
> yaml2obj to create basically any ELF object they want, primarily for
> testing, so being able to test corner cases (e.g. missing tags, malformed
> sections etc) is a significant requirement.
>
> James
>
> On Wed, 16 Jan 2019 at 19:13, Armando Montanez <amontanez at google.com>
> wrote:
>
>> The goal of this proposal is to introduce a new type of YAML section for
>> yaml2obj that allows the population of ELF .dynamic entries via a list of
>> tag and value pairs. These entries are interpreted (and potentially
>> validated) before being written to the .dynamic section. The simplest way
>> to satisfy this requirement is for all dynamic entry values to be numeric
>> values. Unfortunately, this inherently prevents entries like DT_SONAME,
>> DT_NEEDED, DT_RPATH, and DT_RUNPATH from being specified alongside dynamic
>> symbols due to the design of yaml2obj.
>>
>>
>> This proposal introduces three ways to input a value for a dynamic entry.
>> For a given dynamic tag, one or more of these methods of setting a value
>> may be permitted. All of these cases are illustrated later with an example.
>>
>>
>> 1. For dynamic entry strings that belong in .dynstr, the string itself
>> can be used as the value for an entry. (ex. DT_SONAME, DT_NEEDED, DT_RPATH,
>> and DT_RUNPATH)
>>
>>
>> 2. A section name can be used in place of an address. In this case, the
>> value of the dynamic entry is the sh_addr of the specified section. (ex.
>> DT_STRTAB, DT_SYMTAB, DT_HASH, DT_RELA, and others)
>>
>>
>> 3. A value can be specified using hexadecimal or decimal (or other bases
>> supported by `StringRef::to_integer()`). (ex. DT_STRSZ, DT_SYMENT,
>> DT_RELAENT, and others)
>>
>>
>> Here's an example to illustrate this design:
>>
>>
>> !ELF
>>
>> FileHeader:
>>
>>  Class:           ELFCLASS64
>>
>>  Data:            ELFDATA2LSB
>>
>>  Type:            ET_DYN
>>
>>  Machine:         EM_X86_64
>>
>> Sections:
>>
>>  - Name: .dynsym
>>
>>    Type: SHT_DYNSYM
>>
>>    Address: 0x1000
>>
>>  - Name: .data
>>
>>    Type: SHT_PROGBITS
>>
>>    Flags: [ SHF_ALLOC, SHF_WRITE ]
>>
>>  - Name: .dynamic
>>
>>    Type: SHT_DYNAMIC
>>
>>    Entries:
>>
>>      - Tag: DT_SONAME
>>
>>        Value: libsomething.so
>>
>>      - Tag: DT_SYMTAB
>>
>>        Value: .dynsym
>>
>>      - Tag: DT_SYMENT
>>
>>        Value: 0x18
>>
>> DynamicSymbols:
>>
>>  Global:
>>
>>    - Name: foo
>>
>>      Type: STT_FUNC
>>
>>      Section: .data
>>
>>    - Name: bar
>>
>>      Type: STT_OBJECT
>>
>>      Section: .data
>>
>>
>> The final section is of type SHT_DYNAMIC, and the "Entries" key
>> illustrates the proposed addition. Walking through the three dynamic
>> entries,
>>
>>
>> 1. DT_SONAME: The value of this entry is a string that will be inserted
>> into the dynamic string table (.dynstr) alongside the symbol names
>> specified in DynamicSymbols. This is possible due to the nature of .dynstr
>> being represented as a StringTableBuilder, and that .dynamic is linked to
>> .dynstr by default. If the .dynamic section had been linked to a section
>> other than .dynstr, the value of this entry would have to be a number (the
>> offset of the string in the linked string table) rather than a string.
>>
>>
>> 2. DT_SYMTAB: This tag may either be a numeric address or a valid section
>> name, and this example illustrates the option of using the name of a
>> section rather than the address. This resolves to 0x1000 since .dynsym is
>> declared with an address of 0x1000. It would have been equally valid to
>> make this entry have a value of 0x1000, but doing so would mean that
>> changes to .dynsym's address would need to be manually updated in the
>> dynamic entry. It's also worth noting that in the case of DT_SYMTAB it
>> wouldn't be too difficult to infer this.
>>
>>
>> 3. DT_SYMENT: This tag is restricted to only having numeric values. This
>> entry could easily be inferred as well.
>>
>>
>> Note that it doesn’t make sense for DT_SYMENT to be any sort of string,
>> so it is restricted to only being populated with a numeric value.
>> Similarly, it doesn’t make sense for the value of DT_SONAME to ever be
>> interpreted as the name of a section. Though at least one input method is
>> required for a given dynamic tag, it’s typically the case that not all
>> three are valid. It should also be possible to specialize upon certain tags
>> for convenience. For example, DT_PLTREL could be specialized to allow “REL”
>> and “RELA” to be used as values rather than requiring the values be entered
>> in hexadecimal. Evaluating the needs for every dynamic tag isn’t within the
>> scope of this proposal, so any tag without a specialization defaults to
>> permitting numeric values or the name of a valid section (that is later
>> converted to an address).
>>
>>
>> Some dynamic tags have strict enough constraints that they can be
>> inferred. This limited set of dynamic tags could treat “Value” an optional
>> field since the value can be inferred from other parts of an ELF file. This
>> isn’t a requirement for me, though it's something I'd certainly like to
>> have.
>>
>>
>> I began working on a patch here, and it will later be updated to reflect
>> the RFC:
>>
>> https://reviews.llvm.org/D56569
>>
>>
>> Best,
>>
>> Armando
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190117/17ec8bbd/attachment-0001.html>