[llvm-dev] [lldb-dev] Trying out lld to link windows binaries (using msvc as a compiler)

Zachary Turner via llvm-dev llvm-dev at lists.llvm.org
Sun Jan 28 14:14:01 PST 2018

Look for this code in lld/coff/pdb.cpp

if (Config->DebugGHashes) {
ArrayRef<GloballyHashedType> Hashes;
std::vector<GloballyHashedType> OwnedHashes;
if (Optional<ArrayRef<uint8_t>> DebugH = getDebugH(File))
Hashes = getHashesFromDebugH(*DebugH);
else {
OwnedHashes = GloballyHashedType::hashTypes(Types);
Hashes = OwnedHashes;

In the else block there, add a log message that says “synthesizing .debug$h
section for “ + Obj->Name

See how many of these you get. When I build chrome + all third party
libraries this way i get about 100, which is small enough to still see
large performance gains.

If you have many 3rd party libraries, it may be necessary to rewrite the
.lib files too, not just the .obj files. Eventually I’ll get around to
implementing all of this as well, as well as better heuristics in lld-link
to disable ghash if it’s going to be slow
On Sun, Jan 28, 2018 at 1:51 PM Leonardo Santagada <santagada at gmail.com>

> Ok I went for kind of middle ground solution, I patch in the obj files,
> but as adding a new section didn't seem to work, I add a "shadow" section,
> by editing the pointer to line number and the virtual size on the .debug$T
> section. Although technically broken, both link.exe and lld-link.exe don't
> seem to mind the alterations and as the shadow .debug$H is not really a
> section anymore (its just some bytes at the end of the file) it doesn't
> change anything else that does matter. With that I could do my first test
> with a subset of our code base, and the results are not good. I found one
> of our sources that break the ghash computation, I will get more info on
> this and post a proper bug report, but I guess its type information that is
> generated only by msvc. The other more alarming problem is that linking is
> way slower with the ghahes... my guess is that we have a bunch of pdb files
> for some third party libraries and calculating those ghashes takes more
> time than actual linking of this small part of the source (it links in 4s
> in both link.exe and lld-link.exe without ghashes).
> On Fri, Jan 26, 2018 at 8:52 PM, Leonardo Santagada <santagada at gmail.com>
> wrote:
>> We don't generate any .lib as those don't work well with incremental
>> linking (and give zero advantages when linking AFAIK), and it would be
>> pretty easy to have a modern format for having a .ghash for multiple files,
>> something simple like size prefixed name and then size prefixed ghash blobs.
>> On Fri, Jan 26, 2018 at 8:44 PM, Zachary Turner <zturner at google.com>
>> wrote:
>>> We considered that early on, but most object files actually end up in
>>> .lib files so unless there were a way to connect the objects in the .lib to
>>> the corresponding .ghash files, this would disable ghash usage for a large
>>> amount of inputs. Supporting both is an option, but it adds a bit of
>>> complexity and I’m not totally convinced it’s worth it
>>> On Fri, Jan 26, 2018 at 11:38 AM Leonardo Santagada <santagada at gmail.com>
>>> wrote:
>>>> it does.
>>>> I just had an epiphany: why not just write a .ghash file and have lld
>>>> read those if they exist for an .obj file?
>>>> Seem much simpler than trying to wire up a 20 year old file format. I
>>>> will try to do this, is something like this acceptable for LLD? The cool
>>>> thing is that I can generate .ghash for .lib or any obj lying around (maybe
>>>> even for pdb in the future).
>>>> On Fri, Jan 26, 2018 at 8:32 PM, Zachary Turner <zturner at google.com>
>>>> wrote:
>>>>> In general, we should be able to accept any MSVC .obj file to LLD.  At
>>>>> the very least, we're not aware of any cases that don't work.
>>>>> Does your MSVC .obj file link fine before you add the .debug$H?
>>>>> On Fri, Jan 26, 2018 at 11:23 AM Leonardo Santagada <
>>>>> santagada at gmail.com> wrote:
>>>>>> Okay, apparently coff2yaml and yaml2coff are not in a great place as
>>>>>> they both don't deal well with the fact that you can have overlapping
>>>>>> sections, which seems to be what clang-cl produces (the .data section
>>>>>> points to the same place as a later section). Which is not a big big
>>>>>> problem for me particularly because msvc doesn't even generate .data
>>>>>> sections in .obj.
>>>>>> I'm trying to put support for .bss sections in both coff2yaml and
>>>>>> yaml2coff... but I still can link just fine with my transformations
>>>>>> clang-cl generated files... what does give me problems is msvc .obj files.
>>>>>> Have you tried to link one of these?
>>>>>> On Fri, Jan 26, 2018 at 8:05 PM, Leonardo Santagada <
>>>>>> santagada at gmail.com> wrote:
>>>>>>> yeah, apparently .bss has a flag of unitialized data that is not
>>>>>>> being respected on the layout of the coff files (it should skip those
>>>>>>> sections) but I dunno what to do with .data as it doesn't have a size.
>>>>>>> (resending as apparently my pastes generated a ton of hidden html
>>>>>>> data and this message hit the mailinglist limit of 100k)
>>>>>>> --
>>>>>>> Leonardo Santagada
>>>>>> --
>>>>>> Leonardo Santagada
>>>> --
>>>> Leonardo Santagada
>> --
>> Leonardo Santagada
> --
> Leonardo Santagada
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180128/4c8deb18/attachment.html>

More information about the llvm-dev mailing list