[PATCH] D26967: Put opt-viewer critical items in parallel

[Brian Cain via llvm-commits]

bcain added a comment.

In https://reviews.llvm.org/D26967#604923, @fhahn wrote:

> Here are some numbers I gathered for analyzing 250 opt.yaml files generated by compiling clang/llvm with the following options: "-O2 -g -fsave-optimization-record -mllvm -pass-remarks -mllvm -pass-remarks-missed". The size of the .opt.yaml files varies between a few kB and a few MB.
>
> I used Python 2.7.6 on Linux with PyYAML and libyaml  on an Intel Xeon with a large number of cores (`time` used to measure the runtime):


Hmm, that's kinda surprising.  Maybe the overhead of the Lock() is bigger when there's more competition.  Could you artificially limit the parallelism (`min(cpu_count(), 4)` or using `cgroups`/`taskset`)?  Or comment out the `Lock`?

The servers I have access to will be mildly more difficult to run libYAML on but I will give it a go.

I used the Python3.6.0b3 source and only "-fsave-optimization-record".  224 files, ranging from ~10k up to 3.4M.


Repository:
  rL LLVM

https://reviews.llvm.org/D26967