[PATCH] D26967: Put opt-viewer critical items in parallel

[Brian Cain via llvm-commits]

bcain created this revision.
bcain added reviewers: anemet, fhahn.
bcain added a subscriber: llvm-commits.
bcain set the repository for this revision to rL LLVM.
Herald added a subscriber: mehdi_amini.

Put opt-viewer critical items in parallel

Requires features from Python 2.7

**Performance**
Below are performance results across various configurations. These were taken on an i5-5200U (dual core + HT). They were taken with a small subset of the YAML output of building Python 3.6.0b3 with LTO+PGO. 60 YAML files.

"multiprocessing" is the current submission contents. "baseline" is as of 544f14c6b2a07a94168df31833dba9dc35fd8289 (I think this is aka r287505).

"ImportError" vs "class<...CLoader>" below are just confirming the expected configuration (with/without CLoader).

  **************************************** MULTIPROCESSING ****************************************
  PyYAML:
  	Traceback (most recent call last):
  	  File "<string>", line 1, in <module>
  	ImportError: cannot import name CLoader
  	Python 2.7.12
  processed input files 55.1229741573 secs
  mapped remarks 0.0160808563232 secs
  file rendering 1.31177091599 sec
  index rendering 0.514272928238 sec
  total time 56.9831430912 secs
  160.52user 1.44system 0:57.39elapsed 282%CPU (0avgtext+0avgdata 157940maxresident)k
  0inputs+7880outputs (0major+204566minor)pagefaults 0swaps
  
  PyYAML+libYAML:
  	<class 'yaml.cyaml.CLoader'>
  	Python 2.7.12
  processed input files 7.43243384361 secs
  mapped remarks 0.0163559913635 secs
  file rendering 1.27356004715 sec
  index rendering 0.503628969193 sec
  total time 9.24395084381 secs
  21.52user 1.39system 0:09.66elapsed 237%CPU (0avgtext+0avgdata 158652maxresident)k
  0inputs+7880outputs (0major+207754minor)pagefaults 0swaps
  
  PyPy/PyYAML:
  	Traceback (most recent call last):
  	  File "<module>", line 1, in <module>
  	ImportError: cannot import name 'CLoader'
  	Python 2.7.12 (aff251e54385, Nov 09 2016, 18:02:49)
  	[PyPy 5.6.0 with GCC 4.8.2]
  processed input files 17.7338540554 secs
  mapped remarks 0.0163650512695 secs
  file rendering 2.23697400093 sec
  index rendering 0.571249961853 sec
  total time 20.5806679726 secs
  57.87user 1.95system 0:21.16elapsed 282%CPU (0avgtext+0avgdata 282260maxresident)k
  0inputs+7328outputs (0major+286354minor)pagefaults 0swaps
  **************************************** BASELINE        ****************************************
  PyYAML:
  	Traceback (most recent call last):
  	  File "<string>", line 1, in <module>
  	ImportError: cannot import name CLoader
  	Python 2.7.12
  94.07user 0.86system 1:35.82elapsed 99%CPU (0avgtext+0avgdata 112204maxresident)k
  0inputs+7880outputs (0major+30906minor)pagefaults 0swaps
  
  PyYAML+libYAML:
  	<class 'yaml.cyaml.CLoader'>
  	Python 2.7.12
  11.52user 0.64system 0:12.57elapsed 96%CPU (0avgtext+0avgdata 109588maxresident)k
  0inputs+7880outputs (0major+29965minor)pagefaults 0swaps
  
  PyPy/PyYAML:
  	Traceback (most recent call last):
  	  File "<module>", line 1, in <module>
  	ImportError: cannot import name 'CLoader'
  	Python 2.7.12 (aff251e54385, Nov 09 2016, 18:02:49)
  	[PyPy 5.6.0 with GCC 4.8.2]
  18.86user 0.97system 0:20.52elapsed 96%CPU (0avgtext+0avgdata 209160maxresident)k
  0inputs+7880outputs (0major+48344minor)pagefaults 0swaps


Repository:
  rL LLVM

https://reviews.llvm.org/D26967

Files:
  utils/opt-viewer/opt-viewer.py

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D26967.78858.patch
Type: text/x-patch
Size: 10417 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20161122/73b2a5ec/attachment.bin>