[llvm] r286942 - [opt-viewer] Add support for libYAML for faster parsing

[Adam Nemet via llvm-commits]

Author: anemet
Date: Tue Nov 15 02:40:51 2016
New Revision: 286942

URL: http://llvm.org/viewvc/llvm-project?rev=286942&view=rev
Log:
[opt-viewer] Add support for libYAML for faster parsing

This results in a speed-up of over 6x on sqlite3.

Before:

$ time -p /org/llvm/utils/opt-viewer/opt-viewer.py ./MultiSource/Applications/sqlite3/CMakeFiles/sqlite3.dir/sqlite3.c.opt.yaml html
  real 415.07
  user 410.00
  sys 4.66

After with libYAML:

$ time -p /org/llvm/utils/opt-viewer/opt-viewer.py ./MultiSource/Applications/sqlite3/CMakeFiles/sqlite3.dir/sqlite3.c.opt.yaml html
  real 63.96
  user 60.03
  sys 3.67

I followed these steps to get libYAML working with PyYAML: http://rmcgibbo.github.io/blog/2013/05/23/faster-yaml-parsing-with-libyaml/

Modified:
    llvm/trunk/utils/opt-viewer/opt-viewer.py

Modified: llvm/trunk/utils/opt-viewer/opt-viewer.py
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/opt-viewer/opt-viewer.py?rev=286942&r1=286941&r2=286942&view=diff
==============================================================================
--- llvm/trunk/utils/opt-viewer/opt-viewer.py (original)
+++ llvm/trunk/utils/opt-viewer/opt-viewer.py Tue Nov 15 02:40:51 2016
@@ -5,9 +5,16 @@ from __future__ import print_function
 desc = '''Generate HTML output to visualize optimization records from the YAML files
 generated with -fsave-optimization-record and -fdiagnostics-show-hotness.
 
-The tools requires PyYAML and Pygments Python packages.'''
+The tools requires PyYAML and Pygments Python packages.
+
+For faster parsing, you may want to use libYAML with PyYAML.'''
 
 import yaml
+# Try to use the C parser.
+try:
+    from yaml import CLoader as Loader
+except ImportError:
+    from yaml import Loader
 import argparse
 import os.path
 import re
@@ -34,6 +41,9 @@ def demangle(name):
 class Remark(yaml.YAMLObject):
     max_hotness = 0
 
+    # Work-around for http://pyyaml.org/ticket/154.
+    yaml_loader = Loader
+
     @classmethod
     def should_display_hotness(cls):
         # If max_hotness is 0 at the end, we assume hotness information is
@@ -273,7 +283,7 @@ file_remarks = dict()
 
 for input_file in args.yaml_files:
     f = open(input_file)
-    docs = yaml.load_all(f)
+    docs = yaml.load_all(f, Loader=Loader)
     for remark in docs:
         # Avoid remarks withoug debug location or if they are duplicated
         if not hasattr(remark, 'DebugLoc') or remark.key in all_remarks: