[llvm-commits] Atom alignment

[Nick Kledzik]

On Oct 18, 2012, at 1:02 PM, Sid Manning wrote:
> On 10/16/12 15:11, Nick Kledzik wrote:
>> 
>> On Oct 15, 2012, at 1:02 PM, Shankar Kalpathi Easwaran wrote:
>> 
>>> Hi Nick,
>>> 
>>> Thanks for your reply.
>>> 
>>> We would need to put data into seperate sections for the optional
>>> optimization pass to work though.
>> What do you mean by "put into separate sections"? Do you mean
>> -fdata-sections where the compiler creates a unique section for each
>> data item? Or do you mean the ELF Reader will need to do something?
>> 
>> In general, lld downplays sections as much as possible. With the Atom
>> mode,l each atom carries all its own attributes. The only time sections
>> come up is if the output format requires certain sections or the atom
>> specifies it requires a custom section, so the Writer places atoms in
>> the sections required.
>> 
>> In other words, the Pass just re-orders atoms. There are no sections to
>> deal with.
>> 
> 
> The re-order of atoms from their original position is a potential problem.  I think this patch is and even the atom's notion of its alignment is flawed once re-ordering or garbage collection happens.
> 
> .data
> 
> .align 1
> .globl align1
> align1:
>  .long 0x11
> 
> .align 4
> .globl align4
> align4:
>  .long 0x44444444
> 
> .align 0x10
> .globl align10
> align10:
>  .long 0x10101010
> 
> YAML output:
>    - name:              align1
>      scope:             global
>      section-choice:    custom-required
>      section-name:      .data
>      content:           [ 11, 00, 00, 00 ]
> 
>    - name:              align4
>      scope:             global
>      section-choice:    custom-required
>      section-name:      .data
>      content:           [ 44, 44, 44, 44, 00, 00, 00, 00, 00, 00, 00, 00 ]
> 
>    - name:              align10
>      scope:             global
>      section-choice:    custom-required
>      section-name:      .data
>      content:           [ 10, 10, 10, 10 ]
> 
> Alignment is only guaranteed by the sequential ordering of the atoms, if either align1 or align4 are removed or moved the alignment for align10 could be wrong.

That is why the DefinedAtom::Alignment is a struct with two fields: powerOf2 and modulus.   The layout engine needs to both those values to choose an address that is compatible with the assembly file.   Note that we want to eventually get to a world where the compiler can create Atoms directly and they will usually have simple alignments.  It is only because of the way existing assembly language and object formats express alignment info that we need this complexity.

Also, the current YAML reader and writer do not handle the alignment info.    Once I get my YAML I/O stuff in, it will be easier to handle complex types like Alignment. 

Also, in the example above, ".data" should not be a custom section.  That should be the default for DefinedAtom::typeData.  

-Nick

> 
> The section's alignment is the alignment of the most restrictive element in the section.  If, as is the case now, the atom's alignment function returns the owning section's alignment large gaps will open in the output executable.
> 
> 
> 
>> 
>>> what are the tools that exist on darwin to get that information ? Does
>>> it use profile based optimization techniques ?
>> Currently just some dtrace scripts that record access patterns. But
>> there is a lot of room for improvement of the tools.
>> 
>> 
>> -Nick
>> 
>> 
>>> How do we get a file based order, as atoms dont have
>>> 
>>> On Mon, Oct 15, 2012 at 2:21 PM, Nick Kledzik <kledzik at apple.com
>>> <mailto:kledzik at apple.com>> wrote:
>>> 
>>>    On Oct 15, 2012, at 11:54 AM, Shankar Kalpathi Easwaran wrote:
>>>>    Hi,
>>>> 
>>>>    I think we will also get gaps in the output if the atoms are
>>>>    processed in a out of order fashion.
>>>> 
>>>>    Example:
>>>> 
>>>>    Say there are two files that are part of the linking process 1.o,
>>>>    2.o
>>>> 
>>>>    1.o has data symbols (a, b)
>>>>    2.o has data symbols (c, d)
>>>> 
>>>>    a, b has alignment requirements of 4 whereas c, d has alignment
>>>>    requirements of 16
>>>> 
>>>>    So when we try to write the output with the following use cases :
>>>> 
>>>>    1) the atoms appear as a,b,c,d the output file appears properly
>>>>    2) the atoms appear as a,c,b,d the output file will have gaps
>>>>    3) whenever there is an atom movement because of symbol
>>>>    resolution, the output file will have gaps too
>>>> 
>>>>    How do we go about without having gaps in the output file
>>>>    consistently and the output file using optimum space ?
>>>    My philosophy is that the default layout should be super simple
>>>    and predictable (e.g. file order, then order within file, so: a,
>>>    b, c, d which means gaps).
>>> 
>>>    But we can write an optional optimization Pass which re-orders the
>>>    data for better packing. On embedded systems packing might be
>>>    normal mode.
>>> 
>>>    On darwin, we have tools to produce "order files" which determine
>>>    working sets of symbols by monitoring use at runtime. By packing
>>>    data symbols based on usage, you get fewer dirty pages at runtime.
>>> 
>>>    -Nick
>>> 
>>>> 
>>>>    Should the sections and the atom offsets be fixed after running
>>>>    it through the Reader.
>>>> 
>>>>    Thanks
>>>> 
>>>>    Shankar Easwaran
>>>>    Qualcomm Innovation Center, Inc. is a member of Code Aurora
>>>>    Forum, hosted by The Linux Foundation
>>>> 
>>>> 
>>>> 
>>>>        -----Original Message-----
>>>>        From: Sid Manning [mailto:sidneym at codeaurora.org
>>>>        <mailto:sidneym at codeaurora.org>]
>>>>        Sent: Monday, October 15, 2012 11:23 AM
>>>>        To: LLVM Commits; Hemant Kulkarni; Michael Spencer; Nick
>>>>        Kledzik; Evandro; Easwaran, Shankar
>>>>        Subject: Atom alignment
>>>> 
>>>> 
>>>>        DefinedAtom's alignment method is applying section alignment
>>>>        constraints to every atom in the chunk and this causes gaps
>>>>        in the output. This change returns alignment only if the atom
>>>>        is a section symbol.
>>>> 
>>>> 
>>>>        --
>>>>        Qualcomm Innovation Center, Inc. is a member of Code Aurora
>>>>        Forum, hosted by The Linux Foundation
>>>> 
>>>> 
>>> 
>>> 
>> 
> 
> 
> -- 
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation