[all-commits] [llvm/llvm-project] 08e8bb: Revert "[CSSPGO][llvm-profgen] Aggregate samples o...

[ictwanglei via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 08e8bb60cf99fa86890603f99d63a6b854a8ade4
      https://github.com/llvm/llvm-project/commit/08e8bb60cf99fa86890603f99d63a6b854a8ade4
  Author: wlei <wlei at fb.com>
  Date:   2021-02-03 (Wed, 03 Feb 2021)

  Changed paths:
    M llvm/tools/llvm-profgen/PerfReader.cpp
    M llvm/tools/llvm-profgen/PerfReader.h
    M llvm/tools/llvm-profgen/ProfiledBinary.cpp
    M llvm/tools/llvm-profgen/ProfiledBinary.h

  Log Message:
  -----------
  Revert "[CSSPGO][llvm-profgen] Aggregate samples on call frame trie to speed up profile generation"

This reverts commit 1714ad2336293f351b15dd4b518f9e8618ec38f2.


  Commit: 6bccdcdb350d7d33c6eb864d041e05119b5003cb
      https://github.com/llvm/llvm-project/commit/6bccdcdb350d7d33c6eb864d041e05119b5003cb
  Author: wlei <wlei at fb.com>
  Date:   2021-02-03 (Wed, 03 Feb 2021)

  Changed paths:
    R llvm/test/tools/llvm-profgen/Inputs/recursion-compression-noprobe.perfbin
    R llvm/test/tools/llvm-profgen/Inputs/recursion-compression-noprobe.perfscript
    R llvm/test/tools/llvm-profgen/Inputs/recursion-compression-pseudoprobe.perfbin
    R llvm/test/tools/llvm-profgen/Inputs/recursion-compression-pseudoprobe.perfscript
    R llvm/test/tools/llvm-profgen/recursion-compression-noprobe.test
    R llvm/test/tools/llvm-profgen/recursion-compression-pseudoprobe.test
    M llvm/tools/llvm-profgen/PerfReader.cpp
    M llvm/tools/llvm-profgen/ProfileGenerator.cpp
    M llvm/tools/llvm-profgen/ProfileGenerator.h
    M llvm/tools/llvm-profgen/ProfiledBinary.cpp
    M llvm/tools/llvm-profgen/ProfiledBinary.h
    M llvm/tools/llvm-profgen/PseudoProbe.cpp
    M llvm/tools/llvm-profgen/PseudoProbe.h
    M llvm/unittests/tools/CMakeLists.txt
    R llvm/unittests/tools/llvm-profgen/CMakeLists.txt
    R llvm/unittests/tools/llvm-profgen/ContextCompressionTest.cpp

  Log Message:
  -----------
  Revert "[CSSPGO][llvm-profgen] Compress recursive cycles in calling context"

This reverts commit 0609f257dc2e2c3e4c7cd30fe2ffd520117e706b.


  Commit: ac14bb14e7e4e6f4c5a23f4db1ce2d3123a9a24a
      https://github.com/llvm/llvm-project/commit/ac14bb14e7e4e6f4c5a23f4db1ce2d3123a9a24a
  Author: wlei <wlei at fb.com>
  Date:   2021-02-03 (Wed, 03 Feb 2021)

  Changed paths:
    A llvm/test/tools/llvm-profgen/Inputs/recursion-compression-noprobe.perfbin
    A llvm/test/tools/llvm-profgen/Inputs/recursion-compression-noprobe.perfscript
    A llvm/test/tools/llvm-profgen/Inputs/recursion-compression-pseudoprobe.perfbin
    A llvm/test/tools/llvm-profgen/Inputs/recursion-compression-pseudoprobe.perfscript
    A llvm/test/tools/llvm-profgen/recursion-compression-noprobe.test
    A llvm/test/tools/llvm-profgen/recursion-compression-pseudoprobe.test
    M llvm/tools/llvm-profgen/PerfReader.cpp
    M llvm/tools/llvm-profgen/ProfileGenerator.cpp
    M llvm/tools/llvm-profgen/ProfileGenerator.h
    M llvm/tools/llvm-profgen/ProfiledBinary.cpp
    M llvm/tools/llvm-profgen/ProfiledBinary.h
    M llvm/tools/llvm-profgen/PseudoProbe.cpp
    M llvm/tools/llvm-profgen/PseudoProbe.h
    M llvm/unittests/tools/CMakeLists.txt
    A llvm/unittests/tools/llvm-profgen/CMakeLists.txt
    A llvm/unittests/tools/llvm-profgen/ContextCompressionTest.cpp

  Log Message:
  -----------
  [CSSPGO][llvm-profgen] Compress recursive cycles in calling context

This change compresses the context string by removing cycles due to recursive function for CS profile generation. Removing recursion cycles is a way to normalize the calling context which will be better for the sample aggregation and also make the context promoting deterministic.
Specifically for implementation, we recognize adjacent repeated frames as cycles and deduplicated them through multiple round of iteration.
For example:
Considering a input context string stack:
[“a”, “a”, “b”, “c”, “a”, “b”, “c”, “b”, “c”, “d”]
For first iteration,, it removed all adjacent repeated frames of size 1:
[“a”, “b”, “c”, “a”, “b”, “c”, “b”, “c”, “d”]
For second iteration, it removed all adjacent repeated frames of size 2:
[“a”, “b”, “c”, “a”, “b”, “c”, “d”]
So in the end, we get compressed output:
[“a”, “b”, “c”, “d”]

Compression will be called in two place: one for sample's context key right after unwinding, one is for the eventual context string id in the ProfileGenerator.
Added a switch `compress-recursion` to control the size of duplicated frames, default -1 means no size limit.
Added unit tests and regression test for this.

Differential Revision: https://reviews.llvm.org/D93556


Compare: https://github.com/llvm/llvm-project/compare/a2c1054c303f...ac14bb14e7e4