[PATCH] D86295: [analyzer] Reorder the layout of MemRegion and cache by hand for optimal size

[Valeriy Savchenko via Phabricator via cfe-commits]

vsavchenko added a comment.

In D86295#2233157 <https://reviews.llvm.org/D86295#2233157>, @steakhal wrote:

> In D86295#2233076 <https://reviews.llvm.org/D86295#2233076>, @vsavchenko wrote:
>
>> In D86295#2231760 <https://reviews.llvm.org/D86295#2231760>, @NoQ wrote:
>>
>>> I believe @vsavchenko's docker thingy already knows how to do that.
>>
>> Yep, it sure does!  Additionally, there is a `benchmark` subcommand that can chart memory consumption for measured projects.
>
> Could you point me to some docs to help getting started? I haven't used this docker image.

I procrastinated the part about docs 😅 I will add those in the nearest future.

Here is a short summary how to do regression testing (check that all warnings are the same):

1. `cd clang/utils/analyzer`
2. Build test system's docker image (I assume that `docker` is installed): `./SATest.py docker --build-image`
3. Checkout commit that you want to compare against: `git checkout origin/master` or `git checkout HEAD^1`
4. Build LLVM in docker for Ubuntu (if you are on Linux you might not it): `./SATest.py docker --clang-dir {PATH_WHERE_IT_WILL_BE_INSTALLED} --build-dir {PATH_FOR_BUILD_DIR} -- --build-llvm-only`.  I usually put `--build-dir` next to my native build directories, ie `${LLVM_DIR}/build/Docker`
5. Gather reference results: `./SATest.py docker --clang-dir {PATH_WHERE_LLVM_IS_INSTALLED} --build-dir {PATH_FOR_BUILD_DIR} -- build -j1 -r`
6. Checkout your commit that you want to test: Checkout commit that you want to compare against: `git checkout master` or similar
7. Build LLVM with your patch: `./SATest.py docker --clang-dir {PATH_WHERE_IT_WILL_BE_INSTALLED} --build-dir {PATH_FOR_BUILD_DIR} -- --build-llvm-only`.  If you choose the same `--build-dir` it will get compiled incrementally and save you a lot of time.  TIP: you can choose another directory for `--clang-dir` so you don't need to rebuild previous version if you have to re-run it.
8. Gather new results and compare: `./SATest.py docker --clang-dir {PATH_WHERE_LLVM_IS_INSTALLED} --build-dir {PATH_FOR_BUILD_DIR} -- build -j1 --strictness 2`.  NOTE: `SATest.py docker ... -- build` command accepts a comma-separated list of projects for the analysis, so you can choose which projects to test.

Benchmarking is very similar in it's nature, run original, run modified, compare:

5. `./SATest.py docker --clang-dir {PATH_WHERE_LLVM_IS_INSTALLED} --build-dir {PATH_FOR_BUILD_DIR} -- benchmark -o old.csv`
6. ...
7. ...
8. `./SATest.py docker --clang-dir {PATH_WHERE_LLVM_IS_INSTALLED} --build-dir {PATH_FOR_BUILD_DIR} -- benchmark -o new.csv`
9. `./SATest.py docker --clang-dir {PATH_WHERE_LLVM_IS_INSTALLED} --build-dir {PATH_FOR_BUILD_DIR} -- benchmark compare --old old.csv --new new.csv -o comparison.png`

Benchmarks can take a while (they run analysis on the same projects multiple times), so it is recommended to use it for smaller projects.  I will merge D83942 <https://reviews.llvm.org/D83942> and after it you can change commands this way: `./SATest.py docker ... benchmark --max-size small -o blabla.csv`

If you want to see `--help` for a subcommand, drop `docker` and all the options and simply run `./SATest.py build --help` or `./SATest.py benchmark --help`.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D86295/new/

https://reviews.llvm.org/D86295