[PATCH] D49362: [ThinLTO] Internalize read only globals

[Eugene Leviant via Phabricator via llvm-commits]

evgeny777 updated this revision to Diff 169852.
evgeny777 added a comment.

Finally I have some time to work on this again
I've simplified a patch to a large extent and tested it on a number of real applications (see below)

**Algorithm description**

Basically this patch does the following:

- Computes constant references from all functions seen on analysis phase

- Implements constant propagation in combined summary, which allows calculating variables which are never changed in the program

- On the beginning promote phase marks such variables with a special attribute ("thinlto-immutable") in IR if it is possible to internalize them.

- After the end of the import phase internalizes all variables with thinlto-immutable attribute in IR

**Key features:**

- While computing function references patch postpones processing non-volatile load instructions till all other instructions have been processed. This allows grouping constant refs in the end of RefEdges array and we need only 32 bit unsigned integer value per each function summary to represent all constant references in it.

- Patch can internalize both promoted static variables and variables with external or linkonce linkage. Comdats are not currently supported, but can be done as well.

- BC files after each of the stages (promote/internalize/import e.t.c) are consistent and can be sent to legacy builder for further processing.

**Testing**

I've tested it on some of real world applications:

**LLVM testsuite**

Tests pass with following code size improvements/regressions:

F7412493: Selection_002.png <https://reviews.llvm.org/F7412493>
F7412494: Selection_003.png <https://reviews.llvm.org/F7412494>

Size regressions are caused by more agressive inlining caused by lower inlining costs
due to constant folding. Can provide size analysis data on request

**LLVM/clang**

LIT tests pass on newly built toolchain. Unfortunately only few variables were internalized,
so no noticable impact was observed

**Mozilla Firefox**

Produced working image and `make check` shows all tests passed.

**Chromium**

Produced working image of both `chrome` and `unit_tests`. Some tests fail, but I see no difference
with an image obtained from "regular" build.

**Other**

I observed 1-2% raise of performance on some of our in-house projects

**Missing features**

- ThinLTO incremental build *may* fail with link error. To avoid this `HasComdat` flag has to be added to `GlobalValueSummary` and used in module hash computation.

- Some Bitcode tests fail due to version bump and extra field in `FS_PERMODULE` and `FS_COMBINED`

- Patch lacks tests for various corner cases

This won't take long to fix, but I'd appreciate some feedback for overall approach, while I'm doing this


https://reviews.llvm.org/D49362

Files:
  include/llvm/IR/ModuleSummaryIndex.h
  include/llvm/Transforms/IPO/FunctionImport.h
  include/llvm/Transforms/Utils/FunctionImportUtils.h
  lib/Analysis/ModuleSummaryAnalysis.cpp
  lib/Bitcode/Reader/BitcodeReader.cpp
  lib/Bitcode/Writer/BitcodeWriter.cpp
  lib/IR/ModuleSummaryIndex.cpp
  lib/LTO/LTO.cpp
  lib/LTO/ThinLTOCodeGenerator.cpp
  lib/Linker/IRMover.cpp
  lib/Transforms/IPO/FunctionImport.cpp
  lib/Transforms/Utils/FunctionImportUtils.cpp
  test/Bitcode/summary_version.ll
  test/Bitcode/thinlto-alias.ll
  test/Bitcode/thinlto-function-summary-callgraph-relbf.ll
  test/Bitcode/thinlto-function-summary-callgraph.ll
  test/LTO/Resolution/X86/dead-strip-alias.ll
  test/ThinLTO/X86/Inputs/globals-import.ll
  test/ThinLTO/X86/Inputs/index-const-prop.ll
  test/ThinLTO/X86/dot-dumper.ll
  test/ThinLTO/X86/globals-import-const-fold.ll
  test/ThinLTO/X86/globals-import.ll
  test/ThinLTO/X86/index-const-prop.ll
  test/ThinLTO/X86/index-const-prop2.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D49362.169852.patch
Type: text/x-patch
Size: 42664 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20181016/09cda269/attachment.bin>