[all-commits] [llvm/llvm-project] 5e4724: [BOLT][NFC] Add timers for MetadataManager invocat...

Wed Sep 4 14:31:48 PDT 2024

  Branch: refs/heads/users/shawbyoung/spr/bolt-match-blocks-with-pseudo-probes
  Home:   https://github.com/llvm/llvm-project
  Commit: 5e47249c00c6f0825c19496e628d1f31d56894c9
      https://github.com/llvm/llvm-project/commit/5e47249c00c6f0825c19496e628d1f31d56894c9
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M bolt/lib/Rewrite/RewriteInstance.cpp
    A bolt/test/timers.c

  Log Message:
  -----------
  [BOLT][NFC] Add timers for MetadataManager invocations

Test Plan: added bolt/test/timers.c

Reviewers: ayermolo, maksfb, rafaelauler, dcci

Reviewed By: dcci

Pull Request: https://github.com/llvm/llvm-project/pull/101267

  Commit: 3902effbfc181bdac5e2131e8583dca99a33d573
      https://github.com/llvm/llvm-project/commit/3902effbfc181bdac5e2131e8583dca99a33d573
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M bolt/lib/Rewrite/PseudoProbeRewriter.cpp
    M llvm/include/llvm/MC/MCPseudoProbe.h
    M llvm/lib/MC/MCPseudoProbe.cpp

  Log Message:
  -----------
  [MC][NFC] Count pseudo probes and function records

Pre-parse pseudo probes section counting the number of probes and
function records. These numbers are used in follow-up diff to
pre-allocate vectors for decoded probes and inline tree nodes.

Additional benefit is avoiding error handling during parsing.

This pre-parsing is fast: for a 404MiB .pseudo_probe section with
43373881 probes and 25228770 function records, it only takes 0.68±0.01s.
The total time of buildAddress2ProbeMap is 21s.

Reviewers: dcci, maksfb, rafaelauler, wlei-llvm, ayermolo

Reviewed By: wlei-llvm

Pull Request: https://github.com/llvm/llvm-project/pull/102774

  Commit: d20d4d6598c3546be964a2df638f4418645bc0b7
      https://github.com/llvm/llvm-project/commit/d20d4d6598c3546be964a2df638f4418645bc0b7
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M llvm/include/llvm/MC/MCPseudoProbe.h

  Log Message:
  -----------
  [MC][NFC] Drop unused MCDecodedPseudoProbeInlineTree::ChildrenToProcess (#100576)

The usage was removed in 3f97016857b0305294f3a55ea220884fb50ce033.

Results in a slight peak RSS reduction in
`perf2bolt --profile-use-pseudo-probes` from 17.24 to 16.85 GiB.

  Commit: a857d324de090fe9723e999eadc6cb29d8141a93
      https://github.com/llvm/llvm-project/commit/a857d324de090fe9723e999eadc6cb29d8141a93
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M llvm/include/llvm/MC/MCPseudoProbe.h
    M llvm/tools/llvm-profgen/ProfiledBinary.cpp
    M llvm/tools/llvm-profgen/ProfiledBinary.h

  Log Message:
  -----------
  [profgen][NFC] Pass parameter as const_ref

Pass `ProbeNode` parameter of `trackInlineesOptimizedAway` as const
reference.

Reviewers: wlei-llvm, WenleiHe

Reviewed By: WenleiHe

Pull Request: https://github.com/llvm/llvm-project/pull/102787

  Commit: cddea6a015b94140e96dee4d0fa902f8536c0a81
      https://github.com/llvm/llvm-project/commit/cddea6a015b94140e96dee4d0fa902f8536c0a81
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M bolt/lib/Rewrite/PseudoProbeRewriter.cpp
    M llvm/include/llvm/MC/MCPseudoProbe.h
    M llvm/lib/MC/MCPseudoProbe.cpp
    M llvm/tools/llvm-profgen/ProfileGenerator.cpp
    M llvm/tools/llvm-profgen/ProfiledBinary.cpp

  Log Message:
  -----------
  [MC][NFC] Statically allocate storage for decoded pseudo probes and function records

Use #102774 to allocate storage for decoded probes (`PseudoProbeVec`)
and function records (`InlineTreeVec`).

Leverage that to also shrink sizes of `MCDecodedPseudoProbe`:
- Drop Guid since it's accessible via `InlineTree`.

`MCDecodedPseudoProbeInlineTree`:
- Keep track of probes and inlinees using `ArrayRef`s now that probes
  and function records belonging to the same function are allocated
  contiguously.

This reduces peak RSS from 13.7 GiB to 9.7 GiB and pseudo probe parsing
time (as part of perf2bolt) from 15.3s to 9.6s for a large binary with
400MiB .pseudo_probe section containing 43M probes and 25M function
records.

Depends on:
#102774
#102787
#102788

Reviewers: maksfb, rafaelauler, dcci, ayermolo, wlei-llvm

Reviewed By: wlei-llvm

Pull Request: https://github.com/llvm/llvm-project/pull/102789

  Commit: 9746055b0a1ae1e7c6aff50fc217dc216605c277
      https://github.com/llvm/llvm-project/commit/9746055b0a1ae1e7c6aff50fc217dc216605c277
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M llvm/lib/MC/MCPseudoProbe.cpp
    M llvm/tools/llvm-profgen/ProfileGenerator.cpp

  Log Message:
  -----------
  [MC][profgen][NFC] Expand auto for MCDecodedPseudoProbe

Expand autos in select places in preparation to #102789.

Reviewers: dcci, maksfb, WenleiHe, rafaelauler, ayermolo, wlei-llvm

Reviewed By: WenleiHe, wlei-llvm

Pull Request: https://github.com/llvm/llvm-project/pull/102788

  Commit: 3dcef4813afc966aa7bb73d733556c369d3a8011
      https://github.com/llvm/llvm-project/commit/3dcef4813afc966aa7bb73d733556c369d3a8011
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M bolt/lib/Profile/DataAggregator.cpp
    M bolt/lib/Profile/YAMLProfileWriter.cpp
    M bolt/lib/Rewrite/PseudoProbeRewriter.cpp
    M llvm/include/llvm/MC/MCPseudoProbe.h
    M llvm/lib/MC/MCPseudoProbe.cpp
    M llvm/tools/llvm-profgen/ProfileGenerator.cpp

  Log Message:
  -----------
  [MC][NFC] Reduce Address2ProbesMap size

Replace the map from addresses to list of probes with a flat vector
containing probe references sorted by their addresses.

Reduces pseudo probe parsing time from 9.56s to 8.59s and peak RSS from
9.66 GiB to 9.08 GiB as part of perf2bolt processing a large binary.

Test Plan:
```
bin/llvm-lit -sv test/tools/llvm-profgen
```

Reviewers: maksfb, rafaelauler, dcci, ayermolo, wlei-llvm

Reviewed By: wlei-llvm

Pull Request: https://github.com/llvm/llvm-project/pull/102904

  Commit: ba149d99c8dc1d813226b660f3d14b5d879a721c
      https://github.com/llvm/llvm-project/commit/ba149d99c8dc1d813226b660f3d14b5d879a721c
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M bolt/lib/Rewrite/PseudoProbeRewriter.cpp
    M llvm/include/llvm/MC/MCPseudoProbe.h
    M llvm/lib/MC/MCPseudoProbe.cpp

  Log Message:
  -----------
  [MC][NFC] Use vector for GUIDProbeFunctionMap

Replace unordered_map with a vector. Pre-parse the section to statically
allocate storage. Use BumpPtrAllocator for FuncName strings, keep
StringRef in FuncDesc.

Reduces peak RSS of pseudo probe parsing from 9.08 GiB to 8.89 GiB as
part of perf2bolt with a large binary.

Test Plan:
```
bin/llvm-lit -sv test/tools/llvm-profgen
```

Reviewers: wlei-llvm, rafaelauler, dcci, maksfb, ayermolo

Reviewed By: wlei-llvm

Pull Request: https://github.com/llvm/llvm-project/pull/102905

  Commit: c35e8acd11e67fb9a5cd0a66ae51066f24df524a
      https://github.com/llvm/llvm-project/commit/c35e8acd11e67fb9a5cd0a66ae51066f24df524a
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-08-28 (Wed, 28 Aug 2024)

  Changed paths:
    M llvm/lib/MC/MCPseudoProbe.cpp

  Log Message:
  -----------
  buildAddress2ProbeMap timers

  Commit: 1c469cf2dd59241b65e07c5f1030af1d371d881b
      https://github.com/llvm/llvm-project/commit/1c469cf2dd59241b65e07c5f1030af1d371d881b
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-09-04 (Wed, 04 Sep 2024)

  Changed paths:
    M bolt/lib/Profile/DataAggregator.cpp
    M bolt/lib/Profile/YAMLProfileReader.cpp
    M bolt/lib/Profile/YAMLProfileWriter.cpp
    M bolt/lib/Rewrite/PseudoProbeRewriter.cpp
    M bolt/test/X86/pseudoprobe-decoding-inline.test

  Log Message:
  -----------
  [BOLT][NFC] Rename profile-use-pseudo-probes

  Commit: 97f81017f04f23a5ec209e89b3800a34868c7c9a
      https://github.com/llvm/llvm-project/commit/97f81017f04f23a5ec209e89b3800a34868c7c9a
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-09-04 (Wed, 04 Sep 2024)

  Changed paths:
    M bolt/include/bolt/Utils/Utils.h
    M bolt/lib/Rewrite/PseudoProbeRewriter.cpp
    M bolt/lib/Utils/Utils.cpp

  Log Message:
  -----------
  [BOLT][NFCI] Strip suffix in getLTOCommonName

Also provide a mechanism to override the list of suffixes to consider.
Override LTOSuffixes for getGUID in pseudo probe parsing.

Test Plan: 

Reviewers:
Subscribers:

Tasks:

Tags:

Differential Revision: https://phabricator.intern.facebook.com/D61857819

Pull Request: https://github.com/llvm/llvm-project/pull/106243

  Commit: e0a705e3f79c40426af9e4decdcac4cab7129cb4
      https://github.com/llvm/llvm-project/commit/e0a705e3f79c40426af9e4decdcac4cab7129cb4
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-09-04 (Wed, 04 Sep 2024)

  Changed paths:
    M bolt/lib/Rewrite/PseudoProbeRewriter.cpp

  Log Message:
  -----------
  [BOLT] Only parse probes for profiled functions in profile-write-pseudo-probes mode

  Commit: 66fe5d50d65cb6bdda52076c4073508be0a5bc60
      https://github.com/llvm/llvm-project/commit/66fe5d50d65cb6bdda52076c4073508be0a5bc60
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-09-04 (Wed, 04 Sep 2024)

  Changed paths:
    M bolt/include/bolt/Profile/ProfileYAMLMapping.h
    M bolt/lib/Profile/DataAggregator.cpp
    M bolt/lib/Profile/YAMLProfileWriter.cpp
    M bolt/test/X86/pseudoprobe-decoding-inline.test

  Log Message:
  -----------
  [BOLT] Add pseudo probe inline tree to YAML profile

To be used for pseudo probe function matching (#100446).

Test Plan: updated pseudoprobe-decoding-inline.test

Pull Request: https://github.com/llvm/llvm-project/pull/107137

  Commit: 36197b175681d07b4704e576fb008cec3cc1e05e
      https://github.com/llvm/llvm-project/commit/36197b175681d07b4704e576fb008cec3cc1e05e
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2024-09-04 (Wed, 04 Sep 2024)

  Changed paths:
    M bolt/include/bolt/Core/BinaryContext.h
    M bolt/lib/Passes/BinaryPasses.cpp
    M bolt/lib/Profile/StaleProfileMatching.cpp
    M bolt/lib/Rewrite/PseudoProbeRewriter.cpp

  Log Message:
  -----------
  Reworked block probe matching

Use new probe ifaces
Get all function probes at once
Drop ProfileUsePseudoProbes
Unify matchWithBlockPseudoProbes
Distinguish exact and loose probe match

Compare: https://github.com/llvm/llvm-project/compare/ba00b22aaec1...36197b175681

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications