[PATCH] D51330: [ThinLTO] Efficiency fix for writing type id records in per-module indexes

[Teresa Johnson via Phabricator via llvm-commits]

tejohnson created this revision.
tejohnson added a reviewer: pcc.
Herald added subscribers: dexonsmith, steven_wu, inglorion, mehdi_amini.

In https://reviews.llvm.org/D49565/r337503, the type id record writing was fixed so that only
referenced type ids were emitted into each per-module index for ThinLTO
distributed builds. However, this still left an efficiency issue: each
per-module index checked all type ids for membership in the referenced
set, yielding O(M*N) performance (M indexes and N type ids).

This patch changes the writer so that when writing per-module indexes
we instead walk the referenced set, and write the corresponding type id
summary. Since the references are GUIDs but the type id map is indexed
by string, we utilize a map from GUID to type id map entry. This map was
already being created for in-process backends (for caching purposes),
so I moved it to the ThinLTO backend base class and pass it to the
bitcode index writer.

For writing combined indexes we don't build this map and fall back to
the old behavior, which minimized the number of callsites into the index
writer that required changes (but added an assert to check that we
always have the new map when writing per-module indexes where the
performance is going to be an issue).

For a large internal application, this reduced the thin link time by
almost 15%.


Repository:
  rL LLVM

https://reviews.llvm.org/D51330

Files:
  include/llvm/Bitcode/BitcodeWriter.h
  include/llvm/IR/ModuleSummaryIndex.h
  lib/Bitcode/Writer/BitcodeWriter.cpp
  lib/LTO/LTO.cpp
  tools/llvm-lto/llvm-lto.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D51330.162777.patch
Type: text/x-patch
Size: 10602 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180827/401b7d47/attachment.bin>