<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Feb 6, 2015 at 5:58 PM, Michael Spencer <span dir="ltr"><<a href="mailto:bigcheesegs@gmail.com" target="_blank">bigcheesegs@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Fri, Feb 6, 2015 at 5:54 PM, Rui Ueyama <<a href="mailto:ruiu@google.com">ruiu@google.com</a>> wrote:<br>
</span><div><div class="h5">> On Fri, Feb 6, 2015 at 5:42 PM, Michael Spencer <<a href="mailto:bigcheesegs@gmail.com">bigcheesegs@gmail.com</a>><br>
> wrote:<br>
>><br>
>> On Fri, Feb 6, 2015 at 5:31 PM, Rui Ueyama <<a href="mailto:ruiu@google.com">ruiu@google.com</a>> wrote:<br>
>> > There are two questions.<br>
>> ><br>
>> > Firstly, do you think the on-disk format needs to compatible with a C++<br>
>> > struct so that we can cast that memory buffer to the struct? That may be<br>
>> > super-fast but that also comes with many limitations. It's hard to<br>
>> > extend,<br>
>> > for example. Every time we want to store variable-length objects we need<br>
>> > to<br>
>> > define string-table-like data structure. And I'm not very sure that it's<br>
>> > fastest -- because mmap'able objects are not very compact on disk, slow<br>
>> > disk<br>
>> > IO could be a bottleneck, if we compare that with more compact file<br>
>> > format.<br>
>> > I believe Protobufs or Thrust are fast enough or even might be faster.<br>
>><br>
>> I'm not sure here. Although I do question if the object files will<br>
>> even need to be read from disk in your standard edit/compile/debug<br>
>> loop or on a build server. I believe we'll need real data to determine<br>
>> this.<br>
>><br>
>> ><br>
>> > Secondly, do you know why we are dumping post-linked object file to<br>
>> > Native<br>
>> > format? If we want to have a different kind of *object* file format, we<br>
>> > would want to have a tool to convert an object file in an existing file<br>
>> > format (say, ELF) to "native", and teach LLD how read from the file.<br>
>> > Currently we are writing a file in the middle of linking process, which<br>
>> > doesn't make sense to me.<br>
>><br>
>> This is an artifact of having the native format before we had any<br>
>> readers. I agree that it's weird and not terribly useful to write to<br>
>> native format in the middle of the link, although I have found it<br>
>> helpful to output yaml. There's no need to be able to read it back in<br>
>> and resume though.<br>
><br>
><br>
> Even for YAML it doesn't make much sense to write it to a file and read it<br>
> back from the file in the middle of the link, do it? I found that being able<br>
> to output YAML is useful too, but round-trip is a different thing. In the<br>
> middle of the process, we have bunch of additional information that doesn't<br>
> exist in input files and doesn't have to be output to the link result.<br>
> Ability to serialize that intermediate result is not useful.<br>
<br>
</div></div>Completely agree here. We should round-trip the input instead.<br></blockquote><div><br></div><div>Let me remove the round-trip passes. I'll send a patch for review, so let's discuss there.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class="HOEnZb"><font color="#888888"><br>
- Michael Spencer<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
><br>
> Shankar, you added these round-trip tests. Do you have any opinion?<br>
><br>
>> Ideally lld -r would be the tool we use to convert COFF/ELF/MachO to<br>
>> the native format.<br>
>><br>
>> - Michael Spencer<br>
>><br>
>> ><br>
>> > On Fri, Feb 6, 2015 at 5:02 PM, Michael Spencer <<a href="mailto:bigcheesegs@gmail.com">bigcheesegs@gmail.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> On Fri, Feb 6, 2015 at 2:54 PM, Rui Ueyama <<a href="mailto:ruiu@google.com">ruiu@google.com</a>> wrote:<br>
>> >> > Can we remove Native format support? I'd like to get input from<br>
>> >> > anyone<br>
>> >> > who<br>
>> >> > wants to keep the current Native format in LLD.<br>
>> >><br>
>> >> One of the original goals for LLD was to provide a new object file<br>
>> >> format for performance. The reason it is not used currently is because<br>
>> >> we've yet to teach llvm to generate it, and we haven't done that<br>
>> >> because it hasn't been finalized yet. The value it currently provides<br>
>> >> is catching stuff like this, so we can fix it now instead of down the<br>
>> >> road when we actually productize the native format.<br>
>> >><br>
>> >> As for the specific implementation of the native format, I'm open to<br>
>> >> an extensible format, but only if the performance cost is low.<br>
>> >><br>
>> >> - Michael Spencer<br>
>> >><br>
>> >> ><br>
>> >> > On Thu, Feb 5, 2015 at 2:03 PM, Shankar Easwaran<br>
>> >> > <<a href="mailto:shankare@codeaurora.org">shankare@codeaurora.org</a>><br>
>> >> > wrote:<br>
>> >> >><br>
>> >> >> The only way currently is to create a new reference, unless we can<br>
>> >> >> think<br>
>> >> >> of adding some target specific metadata information in the Atom<br>
>> >> >> model.<br>
>> >> >><br>
>> >> >> This has come up over and over again, we need something in the Atom<br>
>> >> >> model<br>
>> >> >> to store information that is target specific.<br>
>> >> >><br>
>> >> >> Shankar Easwaran<br>
>> >> >><br>
>> >> >><br>
>> >> >> On 2/5/2015 2:22 PM, Simon Atanasyan wrote:<br>
>> >> >>><br>
>> >> >>> Hi,<br>
>> >> >>><br>
>> >> >>> I need an advice on implementation of a very specific kind of<br>
>> >> >>> relocations<br>
>> >> >>> used by MIPS N64 ABI. As usual the main problem is how to pass<br>
>> >> >>> target<br>
>> >> >>> specific<br>
>> >> >>> data over Native/YAML conversion barrier.<br>
>> >> >>><br>
>> >> >>> In this ABI relocation record r_info field in fact consists of five<br>
>> >> >>> subfields:<br>
>> >> >>> * r_sym - symbol index<br>
>> >> >>> * r_ssym - special symbol<br>
>> >> >>> * r_type3 - third relocation type<br>
>> >> >>> * r_type2 - second relocation type<br>
>> >> >>> * r_type - first relocation type<br>
>> >> >>><br>
>> >> >>> Up to three these relocations applied one by one. The first<br>
>> >> >>> relocation<br>
>> >> >>> uses<br>
>> >> >>> an addendum from the relocation record. Each subsequent relocation<br>
>> >> >>> takes<br>
>> >> >>> as<br>
>> >> >>> its addend the result of the previous operation. Only the final<br>
>> >> >>> operation<br>
>> >> >>> actually modifies the location relocated. The first relocation uses<br>
>> >> >>> as<br>
>> >> >>> a reference symbol specified by the r_sym field. The third<br>
>> >> >>> relocation<br>
>> >> >>> assumes NULL symbol.<br>
>> >> >>><br>
>> >> >>> The most interesting case is the second relocation. It uses the<br>
>> >> >>> special<br>
>> >> >>> symbol value given by the r_ssym field. This field can contain four<br>
>> >> >>> predefined values:<br>
>> >> >>> * RSS_UNDEF - zero value<br>
>> >> >>> * RSS_GP - value of gp symbol<br>
>> >> >>> * RSS_GP0 - gp0 value taken from the .MIPS.options or .reginfo<br>
>> >> >>> section<br>
>> >> >>> * RSS_LOC - address of location being relocated<br>
>> >> >>><br>
>> >> >>> So the problem is how to store these four constants in the<br>
>> >> >>> lld::Reference object.<br>
>> >> >>> The RSS_UNDEF is obviously not a problem. To represent the RSS_GP<br>
>> >> >>> value I<br>
>> >> >>> can<br>
>> >> >>> set an AbsoluteAtom created for the "_gp" as the reference's<br>
>> >> >>> target.<br>
>> >> >>> But<br>
>> >> >>> what<br>
>> >> >>> about RSS_GP0 and RSS_LOC? I am considering the following<br>
>> >> >>> approaches<br>
>> >> >>> but<br>
>> >> >>> cannot<br>
>> >> >>> select the best one:<br>
>> >> >>><br>
>> >> >>> a) Create AbsoluteAtom for each of these cases and set them as the<br>
>> >> >>> reference's target.<br>
>> >> >>> The problem is that these atoms are fake and should not go to<br>
>> >> >>> the<br>
>> >> >>> symbol table.<br>
>> >> >>> One more problem is to select unique names for these atoms.<br>
>> >> >>> b) Use two high bits of lld::Reference::_kindValue field to encode<br>
>> >> >>> RSS_xxx value.<br>
>> >> >>> Then decode these bits in the RelocationHandler to calculate<br>
>> >> >>> result<br>
>> >> >>> of relocation.<br>
>> >> >>> In that case the problem is how to represent a relocation kind<br>
>> >> >>> value in YAML format.<br>
>> >> >>> The simple xxxRelocationStringTable::kindStrings[] array will<br>
>> >> >>> not<br>
>> >> >>> satisfy us.<br>
>> >> >>> c) Add one more field to the lld::Reference class. Something like<br>
>> >> >>> the<br>
>> >> >>> DefinedAtom::CodeModel<br>
>> >> >>> field.<br>
>> >> >>><br>
>> >> >>> Any advices, ideas, and/or objections are much appreciated.<br>
>> >> >>><br>
>> >> >><br>
>> >> >><br>
>> >> >> --<br>
>> >> >> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,<br>
>> >> >> hosted<br>
>> >> >> by the Linux Foundation<br>
>> >> >><br>
>> >> ><br>
>> >> ><br>
>> >> > _______________________________________________<br>
>> >> > LLVM Developers mailing list<br>
>> >> > <a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
>> >> > <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
>> >> ><br>
>> ><br>
>> ><br>
><br>
><br>
</div></div></blockquote></div><br></div></div>