<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><br><div><div>On Mar 15, 2014, at 10:02 PM, Shankar Easwaran <<a href="mailto:shankare@codeaurora.org">shankare@codeaurora.org</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=windows-1252" http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Hi Nick,<br>
<br>
This is the overall design that I am working on to support
reading/resolving section groups in the YAML reader.<br>
<br>
a) Read all atoms<br>
b) any atoms with group-parent references are moved to
AtomList<lld::DefinedAtom> _groupChildAtoms;<br>
c) There would also be a map _groupChildMap that has a key from
groupChild to groupParent, this is to support (d)<br>
d) If there is a reference to a groupChild from another
child(whose parent is different from the previous group child),
the groupChild reference is changed to refer to a undefined atom
for the groupChild<br>
e) if there is a reference to a groupchild that doesnot have a
group-parent reference, and if the reference is not to a atom that
is undefined, create an undefinedatom and point the group child.<br></div></div></blockquote><div>It is not clear if you mean the ELF Reader will be doing this or the SymbolTable. I think any creation of UndefinedAtoms should be done in the ELF Reader to properly present the ELF file as atoms. The SymbolTable/Resolver just coalesce atoms and bind references.</div><div><br></div><br><blockquote type="cite"><div text="#000000" bgcolor="#FFFFFF"><div class="moz-cite-prefix"><b>We could also do something where all groupchild references
refer to undefined atoms.</b><br>
<br>
<b>There seems to be a caveat even after everything looks sane.<br>
<br>
a) There is a funny case of what do we do when the group child
is a local symbol ? So if there is a reference to a group child
whose scope is local, we cannot create a undefined reference to
the group child right ?<br></b></div></div></blockquote><div>From <<a href="http://docs.oracle.com/cd/E23824_01/html/819-0690/chapter7-26.html">http://docs.oracle.com/cd/E23824_01/html/819-0690/chapter7-26.html</a>></div><div><ul style="margin-left: 13px; padding-left: 0px; font-family: Arial, Helvetica, Luxi-sans, 'Nimbus Sans L', sans-serif; font-size: 14px; background-color: rgb(255, 255, 255);"><li style="margin-left: 13px; padding-left: 0px; list-style-image: url(http://docs.oracle.com/cd/E23824_01/html/819-0690/graphics/ul_bullet.gif);"><p style="margin-top: 3px; margin-bottom: 17px;">References to the sections comprising a group from sections outside of the group must be made through symbol table entries with <tt style="font-family: Monaco, Courier, 'Courier New', monospace; color: rgb(68, 68, 68);">STB_GLOBAL</tt> or <tt style="font-family: Monaco, Courier, 'Courier New', monospace; color: rgb(68, 68, 68);">STB_WEAK</tt> binding and section index<tt style="font-family: Monaco, Courier, 'Courier New', monospace; color: rgb(68, 68, 68);">SHN_UNDEF</tt>. A definition of the same symbol in the object containing the reference must have a separate symbol table entry from the reference. Sections outside of the group can not reference symbols with <tt style="font-family: Monaco, Courier, 'Courier New', monospace; color: rgb(68, 68, 68);">STB_LOCAL</tt> binding for addresses that are contained in the group's sections, including symbols with type <tt style="font-family: Monaco, Courier, 'Courier New', monospace; color: rgb(68, 68, 68);">STT_SECTION</tt>.</p></li></ul><div>I read that as references to local symbol in a group can only come from other symbols in the group. It also sounds like the ELF file already have undefined symbols to reference definitions in a group. </div></div><div><br></div><br><blockquote type="cite"><div text="#000000" bgcolor="#FFFFFF"><div class="moz-cite-prefix"><b>b) The problem is we call the RoundTripYAML reader/writer and
the RoundTripNative reader/writer, which happens to be called
after the resolver is called, which would create undefined
references to atoms!.</b><b><br></b></div></div></blockquote><div>The issue is that to support easy replacement, groups are purposely unresolved (non-canonical) in .o files. Normally, that is fine because they will be processed through the Resolver which will replace any undefines yielding a canonical atom graph. But RoundTripYAML (and maybe RoundTripNative) could regenerate the unresolved state which will not be sent through the resolver again, so the Writer or other Passes will see a non-canonical atom graph. </div><div><br></div><div>The same issue arises if the linker is being using to merge .o files (-r mode). The resolver will produce a canonical graph (no extra undefines), but the ELF Writer will want them.</div><div><br></div><div>I think the solution:</div><div>1) keep the existing canonical form for an atom graph that Passes process</div><div>2) make the ELF Writer smart that when it writes relocatable object files, to synthesize undefined symbols for references into group children</div><div>3) Have the YAML Reader be dumb and not synthesize any undefines. Test cases for groups that want undefines will need to explicitly have them in the yaml. This also enables you to write test cases that test the error case where an indirection though an undefine is not used.</div><div><br></div><div><br></div><div>-Nick</div><div><br></div><br><blockquote type="cite"><div text="#000000" bgcolor="#FFFFFF"><div class="moz-cite-prefix"><b>
</b>I am concerned on just the above cases ?<br>
<br>
Do we want to have a different mode in the YAML reader when called
from RoundTripYAMLPass not to create undefined atoms ? <br>
<br>
Am I missing something ?<br>
<br>
Thanks<br>
<br>
Shankar Easwaran<br>
<br>
On 3/14/2014 8:29 PM, Nick Kledzik wrote:<br>
</div>
<blockquote cite="mid:AC057FF5-E7F7-432C-8951-A33725E10A14@apple.com" type="cite">
<pre wrap="">On Mar 14, 2014, at 4:37 PM, Shankar Easwaran <a class="moz-txt-link-rfc2396E" href="mailto:shankare@codeaurora.org"><shankare@codeaurora.org></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On 3/14/2014 5:52 PM, Nick Kledzik wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Mar 13, 2014, at 8:24 PM, Shankar Kalpathi Easwaran <a class="moz-txt-link-rfc2396E" href="mailto:shankarke@gmail.com"><shankarke@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap=""> I dont think the multithreaded model of processing all the object files first is going to work with the ELF linker, as resolution happens in the way input files are processed from command line.
This is what I see with the GNU linker :-
a) The groups are being resolved as and when input files are processed by the resolver. We cannot delay processing the group members after the resolver has processed all files.
b) Consider the below example :-
</pre>
</blockquote>
<pre wrap="">...
</pre>
<blockquote type="cite">
<pre wrap=""> With the command line ld foo.o bar.o libarchive.a, when symbols from bar.o are processed, **global-f3 becomes a undefined symbol** and pulls the global-f3 from libarchive.a
How do we handle all these cases with lld ?
<a class="moz-txt-link-freetext" href="http://llvm-reviews.chandlerc.com/D2961">http://llvm-reviews.chandlerc.com/D2961</a>
</pre>
</blockquote>
<pre wrap="">Given that the ELF linker needs those semantics, it seems that the Resolver and symbol table building must be done serially, in command line order. The object files could still be parsed in parallel, but the results would need to be buffered and added to the symbol table in command line order. Given that, here are two possible algorithms:
1) Groups are like static libraries
a) group child atoms are not returned in the defined() atoms collection, but group parents are.
b) files are added to the symbol table in command line order
c) when a group parent atom is added to the symbol table, its name is checked to see if that group name has been seen yet by the symbol table. If not, the group name is added and all its children are also added. If the group name has already been seen, then the group’s child remain unseen by the symbol table.
Problem: uses of any atom in a group from outside the group would require the ELF reader to synthesized UndefinedAtom and have that be referenced. In a way this is good because it enforces that groups can be removed.
</pre>
</blockquote>
<pre wrap="">Thanks for the great suggestion. We could follow the option (1) above, as the group childs are not seen by the rest of the linker until the group is added.
Are you saying that the group childs are internal to the lld::File, and can be only referenced from the group-child references in the parent atom (contentType:kindGroupComdat) ?
</pre>
</blockquote>
<pre wrap="">Without adding some other mechanism for getting a list of atoms, yes, that is how to do it.
</pre>
<blockquote type="cite">
<pre wrap="">To support groups in the core/yaml reader, the user has to be explicit in creating undefined/defined atoms for the group child members ?
</pre>
</blockquote>
<pre wrap="">I supposed you could have the yaml reader detect and automatically create the UndefinedAtoms. I’m not sure which is easier.
Also, this is going to require making lots of error laden test cases in ELF assembly and see how the existing ELF linkers handle them. For instance if group g1 contains a function that calls foo, and that is the only call to foo in the object file, when that file is loaded, if g1 is not used (because another group named g1 already exists), will the undef for foo get added to the global symbol table? If not, then somehow undefined symbols also need to be associated with groups.
-Nick
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation</pre>
</div>
</blockquote></div><br></body></html>