[cfe-dev] Clang vs MSVC: Smaller objs, but bigger DLLs?

Thu Jan 18 16:48:09 PST 2018

Have you tried /Gw, as detailed in:

https://blogs.msdn.microsoft.com/vcblog/2013/09/11/introducing-gw-compiler-switch/

On Jan 18, 2018 7:43 PM, "Cullen, Colden via cfe-dev" <
cfe-dev at lists.llvm.org> wrote:

> Hey Reid,
>
>
>
> Thanks for the input!
>
>
>
> >  First, there is /Zc:inline. After inlining, clang discards any
> unreferenced inline functions that it can. MSVC doesn't do this as much. In
> theory, the /Zc:inline flag exists to enable this behavior, but I've had
> mixed results.
>
> I spoke with a colleague who’s been working on our build system for a
> while and he mentioned that we have tried /Zc:inline in the past, to
> similarly mixed results. Apparently at one point he even found an edge case
> where it would throw out symbols that were in use and very much required.
> That didn’t go so good… I don’t think this would really solve my problem
> though. At this point I don’t really care about the size of the MSVC object
> files, as it’s not causing a problem. I can link the MSVC-generated object
> files with whatever linker I want and with or without /DEBUG, so at this
> point it ain’t broke.
>
>
>
> > Next, clang generally inlines more than MSVC. This is just part of the
> nature of the bottom-up inliner in LLVM. It lives to remove C++ abstraction
> penalty. This can result in larger final code for obvious reasons.
>
> > Essentially, if you inline less, there will be more duplicate data in
> the object files, which then gets tossed during linking, and then you have
> a smaller binary.
>
> This is interesting, because my Clang object files are smaller than the
> MSVC objects. In debug builds (which is the type I’m running into issues
> with), no -O flags are passed, so presumably it’s not doing any inling, or
> at least nothing too crazy.
>
>
>
> > I would recommend experimenting with /O1, which is equivalent to -Os in
> clang-cl. This is mostly equivalent to -O2 with a lower inlining threshold,
> with some additional codegen tweaks in the backend. This should give you a
> smaller final binary.
>
> I will play around with these and see if I can get my DLL to successfully
> build. Maybe I’ll learn something J
>
>
>
> > With regard to LLD, I think the size differences come from incremental
> linking and whether /OPT:REF and /OPT:ICF are enabled. I think we still
> have inconsistencies there. In link.exe, /debug enables /incremental and
> disables /opt:ref and /opt:icf. LLD doesn't support incremental linking, so
> its /debug output is smaller.
>
> I tried turning both /OPT:REF and /OPT:ICF on, and it definitely reduced
> my final image size, but not enough L I’m still getting that shiny new
> “image too large” error.
>
>
>
> I still can’t wrap my head around why on earth the Clang object files are
> smaller but produce a final image that’s orders of magnitude larger.
> Especially since the .text sections are about the same size across
> compilers, and Clang’s .debug sections are much smaller than MSVC’s. I’ll
> keep poking through objdumps, but if anyone else has any ideas, please
> speak up!
>
>
>
> Thanks
>
> Colden
>
>
>
> *From:* Reid Kleckner [mailto:rnk at google.com]
> *Sent:* Thursday, January 18, 2018 2:54 PM
> *To:* Cullen, Colden <colden at amazon.com>
> *Cc:* cfe-dev at lists.llvm.org
> *Subject:* Re: [cfe-dev] Clang vs MSVC: Smaller objs, but bigger DLLs?
>
>
>
> Cool!
>
>
>
> Regarding your numbers, I can think of a few differences that might be
> relevant.
>
>
>
> First, there is /Zc:inline. After inlining, clang discards any
> unreferenced inline functions that it can. MSVC doesn't do this as much. In
> theory, the /Zc:inline flag exists to enable this behavior, but I've had
> mixed results.
>
>
>
> Next, clang generally inlines more than MSVC. This is just part of the
> nature of the bottom-up inliner in LLVM. It lives to remove C++ abstraction
> penalty. This can result in larger final code for obvious reasons.
> Essentially, if you inline less, there will be more duplicate data in the
> object files, which then gets tossed during linking, and then you have a
> smaller binary.
>
>
>
> I would recommend experimenting with /O1, which is equivalent to -Os in
> clang-cl. This is mostly equivalent to -O2 with a lower inlining threshold,
> with some additional codegen tweaks in the backend. This should give you a
> smaller final binary.
>
>
>
> With regard to LLD, I think the size differences come from incremental
> linking and whether /OPT:REF and /OPT:ICF are enabled. I think we still
> have inconsistencies there. In link.exe, /debug enables /incremental and
> disables /opt:ref and /opt:icf. LLD doesn't support incremental linking, so
> its /debug output is smaller.
>
>
>
> I think this is completely explained in the second table. For debug
> binaries, link is bigger because it is incremental. "link ND" and "lld-link
> D" are almost identical in size. I think this may be because link.exe isn't
> doing ICF, and lld-link /debug disables ICF. Removing /debug and using LLD
> enables ICF and GC, and that explains the smaller binary in the "lld-link
> ND" case.
>
>
>
> ---
>
>
>
> Finally, given how much smaller the binary with clang objects gets in the
> "lld-link ND" case, it sounds like clang is generating a ton of
> unreferenced code and data for your codebase. It's not immediately clear
> what's causing that. You can get some visibility into it by linking with
> /verbose, and that will show you what's being discarded.
>
>
>
> On Thu, Jan 18, 2018 at 11:42 AM, Cullen, Colden via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
> Hey everyone,
>
> First of all, thanks to everyone who's given me feedback on the patches
> I've posted recently! I really appreciate you taking the time to review my
> code.
>
> For context, I work on Amazon Lumberyard, and my current goal is to get
> the entire engine and toolset to build on Windows with Clang and LLD as an
> alternative to MSVC. So far I've made a ton of progress; I've fixed a few
> bugs in LLVM, and fixed an uncountable number of bugs and warnings in our
> codebase that slipped past other compilers.
>
> The most recent issue I've run into is quite perplexing. When building our
> largest legacy DLL, I see the following behavior:
> - The .obj files produced by Clang are about half the size of the
> corresponding MSVC objects (719 files, 17.2GB from MSVC, 8.9GB from Clang)
> - Linking the MSVC set of object files with both linkers with or without
> /DEBUG is successful
> - Linking the Clang set of object files with LLD without /DEBUG is
> successful
> - Linking the Clang set of object files with MSVC's link.exe with or
> without /DEBUG and with LLD with /DEBUG results in a "binary too large"
> error.
>
> Here's a table of linker, Debug or No Debug, and resulting binary size for
> the Clang object files:
> link.exe D:  4297254686 (reported)
> link.exe ND: 4295125179 (reported)
> lld-link D:  5558142976 (reported)
> lld-link ND:   77908000 (on disk)
> MAX SIZE:    4294967295
>
> And here's the results for the MSVC object files:
> link.exe D:   135228416 (on disk)
> link.exe ND:  109041152 (on disk)
> lld-link D:   109095424 (on disk)
> lld-link ND:   84219904 (on disk)
>
> Clang and LLD are built from trunk, and I'm running both linkers with the
> following options:
> /MACHINE:x64 /MANIFEST /LARGEADDRESSAWARE /DEFAULTLIB:msvcrtd
> /SUBSYSTEM:WINDOWS /NODEFAULTLIB:LIBCMTD.lib /DLL /TIME /ignore:4099
> [/DEBUG]
>
> I'm going to start poking around object files to see what I can learn, but
> does anyone have any tips on debugging something like this? I honestly
> don't really know where to start.
>
> If this turns out to be extra troublesome, I can work on getting legal
> approval to share my compiled objects to help you repro it. That can be a
> tricky process, so I'd like to just debug vicariously if possible.
>
> Thanks!
> Colden Cullen
> colden at amazon.com
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180118/3d253b3e/attachment.html>