[PATCH] D37409: [CodeView] Don't output nested typedefs that are scoped to classes.

[Zachary Turner via Phabricator via llvm-commits]

zturner added a comment.

In https://reviews.llvm.org/D37409#861130, @amccarth wrote:

> Nice description of the problem.
>
> I'm surprised the size difference is so dramatic.  It's hard to imagine that many classes/structs having that many typedefs, except for the standard containers.


@amccarth: The problem is two-fold.  First, there is the issue of just the S_UDT records themselves.  You're right that most of them come from standard containers, but there's so many.  Every instantiation, gets one, and since some classes are instantiated with other template classes, you get a large explosion of template instantiations, and thusly a large number of these S_UDT records.  I ran `llvm-pdbutil dump -udt-stats` before and after, and came up with this:

               Patch    No Patch  % Decrease
  S_UDT Count   6,769    89,421     92%
  S_UDT Size   81,936   835,664     90%

So as you can see, the records themselves are only responsible for a size reduction of ~753KB.  But /DEBUG:FASTLINK PDBs don't work the same way as regular PDBs.  Normally the linker would merge all the S_UDT records into a single global symbol stream, eliminating duplicates, and take them out of the module stream, but with a /DEBUG:FASTLINK PDB, all of the symbols stay in the original module stream.  This means you end up with 10s of thousands of duplicate S_UDT records in a /DEBUG:FASTLINK PDB.  So that 835KB size increase, multiply it by a couple of orders of magnitude and there's your size increase.

To prove this, I ran `llvm-pdbutil dump -sym-stats` on the fastlink versions of the before & after PDBs.

  D:\analyze>llvm-pdbutil dump -sym-stats libclang-fastlink-patch.pdb
  
    Summary |
  
        Symbols
                                           Total: 2853675 entries (341483776 bytes)
        --------------------------------------------------------------------------
                                       S_THUNK32:     681 entries (   31356 bytes)
                                       S_SECTION:       8 entries (     224 bytes)
                                      S_ENVBLOCK:       2 entries (     556 bytes)
                                        S_EXPORT:     353 entries (   13872 bytes)
                                    S_TRAMPOLINE:  297459 entries ( 5949180 bytes)
                                           S_END:     681 entries (    2724 bytes)
                                      S_FASTLINK: 2553073 entries (335435108 bytes)
                                       S_OBJNAME:     686 entries (   16552 bytes)
                                      S_COMPILE3:     686 entries (   32928 bytes)
                                     S_COFFGROUP:      46 entries (    1276 bytes)
  
        Chunks
                                           Total:  500501 entries (96025508 bytes)
        --------------------------------------------------------------------------
                              DEBUG_S_FILECHKSMS:    1300 entries (  447104 bytes)
                         DEBUG_S_COFF_SYMBOL_RVA:    1990 entries (51430364 bytes)
                                   DEBUG_S_LINES:  497211 entries (44148040 bytes)
  
  There are 8,891 `S_FASTLINK` records here.  If we do the same on the fastlink PDB that has the patch, we get:

D:\analyze>llvm-pdbutil dump -sym-stats libclang-fastlink-nopatch.pdb

  Symbols
                                     Total: 6521428 entries (909728300 bytes)
  --------------------------------------------------------------------------
                                 S_THUNK32:     681 entries (   31356 bytes)
                                 S_SECTION:       8 entries (     224 bytes)
                                S_ENVBLOCK:       2 entries (     556 bytes)
                                  S_EXPORT:     353 entries (   13872 bytes)
                              S_TRAMPOLINE:  297459 entries ( 5949180 bytes)
                                     S_END:     681 entries (    2724 bytes)
                                S_FASTLINK: 6220826 entries (903679632 bytes)
                                 S_OBJNAME:     686 entries (   16552 bytes)
                                S_COMPILE3:     686 entries (   32928 bytes)
                               S_COFFGROUP:      46 entries (    1276 bytes)
  
  Chunks
                                     Total:  500501 entries (96025412 bytes)
  --------------------------------------------------------------------------
                        DEBUG_S_FILECHKSMS:    1300 entries (  447104 bytes)
                   DEBUG_S_COFF_SYMBOL_RVA:    1990 entries (51430364 bytes)
                             DEBUG_S_LINES:  497211 entries (44147944 bytes)

  So even though we only eliminated 82,652 //merged// `S_UDT` records, when they're not merged, it accounts for a reduction of 3.6 **million** `S_UDT` records totalling 568MB.
  
  @majnemer: It does output an `S_UDT` in that case, but so do we, so our behavior matches.


https://reviews.llvm.org/D37409