<div dir="ltr">FWIW it looks pretty pythonic to me.</div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Jan 11, 2015 at 8:43 PM, Chandler Carruth <span dir="ltr"><<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: chandlerc<br>
Date: Sun Jan 11 22:43:18 2015<br>
New Revision: 225618<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=225618&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=225618&view=rev</a><br>
Log:<br>
Add a new utility script that helps update very simple regression tests.<br>
<br>
This script is currently specific to x86 and limited to use with very<br>
small regression or feature tests using 'llc' and 'FileCheck' in<br>
a reasonably canonical way. It is in no way general purpose or robust at<br>
this point. However, it works quite well for simple examples. Here is<br>
the intended workflow:<br>
<br>
- Make a change that requires updating N test files and M functions'<br>
assertions within those files.<br>
- Stash the change.<br>
- Update those N test files' RUN-lines to look "canonical"[1].<br>
- Refresh the FileCheck lines for either the entire file or select<br>
functions by running this script.<br>
- The script will parse the RUN lines and run the 'llc' binary you<br>
give it according to each line, collecting the asm.<br>
- It will then annotate each function with the appropriate FileCheck<br>
comments to check every instruction from the start of the first<br>
basic block to the last return.<br>
- There will be numerous cases where the script either fails to remove<br>
the old lines, or inserts checks which need to be manually editted,<br>
but the manual edits tend to be deletions or replacements of<br>
registers with FileCheck variables which are fast manual edits.<br>
- A common pattern is to have the script insert complete checking of<br>
every instruction, and then edit it down to only check the relevant<br>
ones.<br>
- Be careful to do all of these cleanups though! The script is<br>
designed to make transferring and formatting the asm output of llc<br>
into a test case fast, it is *not* designed to be authoratitive<br>
about what constitutes a good test!<br>
- Commit the nice fresh baseline of checks.<br>
- Unstash your change and rebuild llc.<br>
- Re-run script to regenerate the FileCheck annotations<br>
- Remember to re-cleanup these annotations!!!<br>
- Check the diff to make sure this is sane, checking the things you<br>
expected it to, and check that the newly updated tests actually pass.<br>
- Profit!<br>
<br>
Also, I'm *terrible* at writing Python, and frankly I didn't spend a lot<br>
of time making this script beautiful or well engineered. But it's useful<br>
to me and may be useful to others so I thought I'd send it out.<br>
<br>
<a href="http://reviews.llvm.org/D5546" target="_blank">http://reviews.llvm.org/D5546</a><br>
<br>
Added:<br>
llvm/trunk/utils/update_llc_test_checks.py (with props)<br>
<br>
Added: llvm/trunk/utils/update_llc_test_checks.py<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/update_llc_test_checks.py?rev=225618&view=auto" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/update_llc_test_checks.py?rev=225618&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/utils/update_llc_test_checks.py (added)<br>
+++ llvm/trunk/utils/update_llc_test_checks.py Sun Jan 11 22:43:18 2015<br>
@@ -0,0 +1,207 @@<br>
+#!/usr/bin/env python2.7<br>
+<br>
+"""A test case update script.<br>
+<br>
+This script is a utility to update LLVM X86 'llc' based test cases with new<br>
+FileCheck patterns. It can either update all of the tests in the file or<br>
+a single test function.<br>
+"""<br>
+<br>
+import argparse<br>
+import itertools<br>
+import string<br>
+import subprocess<br>
+import sys<br>
+import tempfile<br>
+import re<br>
+<br>
+<br>
+def llc(args, cmd_args, ir):<br>
+ with open(ir) as ir_file:<br>
+ stdout = subprocess.check_output(args.llc_binary + ' ' + cmd_args,<br>
+ shell=True, stdin=ir_file)<br>
+ return stdout<br>
+<br>
+<br>
+ASM_SCRUB_WHITESPACE_RE = re.compile(r'(?!^(| \w))[ \t]+', flags=re.M)<br>
+ASM_SCRUB_SHUFFLES_RE = (<br>
+ re.compile(<br>
+ r'^(\s*\w+) [^#\n]+#+ ((?:[xyz]mm\d+|mem) = .*)$',<br>
+ flags=re.M))<br>
+ASM_SCRUB_SP_RE = re.compile(r'\d+\(%(esp|rsp)\)')<br>
+ASM_SCRUB_RIP_RE = re.compile(r'[.\w]+\(%rip\)')<br>
+ASM_SCRUB_KILL_COMMENT_RE = re.compile(r'^ *#+ +kill:.*\n')<br>
+<br>
+<br>
+def scrub_asm(asm):<br>
+ # Scrub runs of whitespace out of the assembly, but leave the leading<br>
+ # whitespace in place.<br>
+ asm = ASM_SCRUB_WHITESPACE_RE.sub(r' ', asm)<br>
+ # Expand the tabs used for indentation.<br>
+ asm = string.expandtabs(asm, 2)<br>
+ # Detect shuffle asm comments and hide the operands in favor of the comments.<br>
+ asm = ASM_SCRUB_SHUFFLES_RE.sub(r'\1 {{.*#+}} \2', asm)<br>
+ # Generically match the stack offset of a memory operand.<br>
+ asm = ASM_SCRUB_SP_RE.sub(r'{{[0-9]+}}(%\1)', asm)<br>
+ # Generically match a RIP-relative memory operand.<br>
+ asm = ASM_SCRUB_RIP_RE.sub(r'{{.*}}(%rip)', asm)<br>
+ # Strip kill operands inserted into the asm.<br>
+ asm = ASM_SCRUB_KILL_COMMENT_RE.sub('', asm)<br>
+ return asm<br>
+<br>
+<br>
+def main():<br>
+ parser = argparse.ArgumentParser(description=__doc__)<br>
+ parser.add_argument('-v', '--verbose', action='store_true',<br>
+ help='Show verbose output')<br>
+ parser.add_argument('--llc-binary', default='llc',<br>
+ help='The "llc" binary to use to generate the test case')<br>
+ parser.add_argument(<br>
+ '--function', help='The function in the test file to update')<br>
+ parser.add_argument('tests', nargs='+')<br>
+ args = parser.parse_args()<br>
+<br>
+ run_line_re = re.compile('^\s*;\s*RUN:\s*(.*)$')<br>
+ ir_function_re = re.compile('^\s*define\s+(?:internal\s+)?[^@]*@(\w+)\s*\(')<br>
+ asm_function_re = re.compile(<br>
+ r'^_?(?P<f>[^:]+):[ \t]*#+[ \t]*@(?P=f)\n[^:]*?'<br>
+ r'(?P<body>^##?[ \t]+[^:]+:.*?)\s*'<br>
+ r'^\s*(?:[^:\n]+?:\s*\n\s*\.size|\.cfi_endproc|\.globl|\.(?:sub)?section)',<br>
+ flags=(re.M | re.S))<br>
+ check_prefix_re = re.compile('--check-prefix=(\S+)')<br>
+ check_re = re.compile(r'^\s*;\s*([^:]+?)(?:-NEXT|-NOT|-DAG|-LABEL)?:')<br>
+<br>
+ for test in args.tests:<br>
+ if args.verbose:<br>
+ print >>sys.stderr, 'Scanning for RUN lines in test file: %s' % (test,)<br>
+ with open(test) as f:<br>
+ test_lines = [l.rstrip() for l in f]<br>
+<br>
+ run_lines = [m.group(1)<br>
+ for m in [run_line_re.match(l) for l in test_lines] if m]<br>
+ if args.verbose:<br>
+ print >>sys.stderr, 'Found %d RUN lines:' % (len(run_lines),)<br>
+ for l in run_lines:<br>
+ print >>sys.stderr, ' RUN: ' + l<br>
+<br>
+ checks = []<br>
+ for l in run_lines:<br>
+ (llc_cmd, filecheck_cmd) = tuple([cmd.strip() for cmd in l.split('|', 1)])<br>
+ if not llc_cmd.startswith('llc '):<br>
+ print >>sys.stderr, 'WARNING: Skipping non-llc RUN line: ' + l<br>
+ continue<br>
+<br>
+ if not filecheck_cmd.startswith('FileCheck '):<br>
+ print >>sys.stderr, 'WARNING: Skipping non-FileChecked RUN line: ' + l<br>
+ continue<br>
+<br>
+ llc_cmd_args = llc_cmd[len('llc'):].strip()<br>
+ llc_cmd_args = llc_cmd_args.replace('< %s', '').replace('%s', '').strip()<br>
+<br>
+ check_prefixes = [m.group(1)<br>
+ for m in check_prefix_re.finditer(filecheck_cmd)]<br>
+ if not check_prefixes:<br>
+ check_prefixes = ['CHECK']<br>
+<br>
+ # FIXME: We should use multiple check prefixes to common check lines. For<br>
+ # now, we just ignore all but the last.<br>
+ checks.append((check_prefixes, llc_cmd_args))<br>
+<br>
+ asm = {}<br>
+ for prefixes, _ in checks:<br>
+ for prefix in prefixes:<br>
+ asm.update({prefix: dict()})<br>
+ for prefixes, llc_args in checks:<br>
+ if args.verbose:<br>
+ print >>sys.stderr, 'Extracted LLC cmd: llc ' + llc_args<br>
+ print >>sys.stderr, 'Extracted FileCheck prefixes: ' + str(prefixes)<br>
+ raw_asm = llc(args, llc_args, test)<br>
+ # Build up a dictionary of all the function bodies.<br>
+ for m in asm_function_re.finditer(raw_asm):<br>
+ if not m:<br>
+ continue<br>
+ f = m.group('f')<br>
+ f_asm = scrub_asm(m.group('body'))<br>
+ if args.verbose:<br>
+ print >>sys.stderr, 'Processing asm for function: ' + f<br>
+ for l in f_asm.splitlines():<br>
+ print >>sys.stderr, ' ' + l<br>
+ for prefix in prefixes:<br>
+ if f in asm[prefix] and asm[prefix][f] != f_asm:<br>
+ if prefix == prefixes[-1]:<br>
+ print >>sys.stderr, ('WARNING: Found conflicting asm under the '<br>
+ 'same prefix!')<br>
+ else:<br>
+ asm[prefix][f] = None<br>
+ continue<br>
+<br>
+ asm[prefix][f] = f_asm<br>
+<br>
+ is_in_function = False<br>
+ is_in_function_start = False<br>
+ prefix_set = set([prefix for prefixes, _ in checks for prefix in prefixes])<br>
+ if args.verbose:<br>
+ print >>sys.stderr, 'Rewriting FileCheck prefixes: %s' % (prefix_set,)<br>
+ fixed_lines = []<br>
+ for l in test_lines:<br>
+ if is_in_function_start:<br>
+ if l.lstrip().startswith(';'):<br>
+ m = check_re.match(l)<br>
+ if not m or m.group(1) not in prefix_set:<br>
+ fixed_lines.append(l)<br>
+ continue<br>
+<br>
+ # Print out the various check lines here<br>
+ printed_prefixes = []<br>
+ for prefixes, _ in checks:<br>
+ for prefix in prefixes:<br>
+ if prefix in printed_prefixes:<br>
+ break<br>
+ if not asm[prefix][name]:<br>
+ continue<br>
+ if len(printed_prefixes) != 0:<br>
+ fixed_lines.append(';')<br>
+ printed_prefixes.append(prefix)<br>
+ fixed_lines.append('; %s-LABEL: %s:' % (prefix, name))<br>
+ asm_lines = asm[prefix][name].splitlines()<br>
+ fixed_lines.append('; %s: %s' % (prefix, asm_lines[0]))<br>
+ for asm_line in asm_lines[1:]:<br>
+ fixed_lines.append('; %s-NEXT: %s' % (prefix, asm_line))<br>
+ break<br>
+ is_in_function_start = False<br>
+<br>
+ if is_in_function:<br>
+ # Skip any blank comment lines in the IR.<br>
+ if l.strip() == ';':<br>
+ continue<br>
+ # And skip any CHECK lines. We'll build our own.<br>
+ m = check_re.match(l)<br>
+ if m and m.group(1) in prefix_set:<br>
+ continue<br>
+ # Collect the remaining lines in the function body and look for the end<br>
+ # of the function.<br>
+ fixed_lines.append(l)<br>
+ if l.strip() == '}':<br>
+ is_in_function = False<br>
+ continue<br>
+<br>
+ fixed_lines.append(l)<br>
+<br>
+ m = ir_function_re.match(l)<br>
+ if not m:<br>
+ continue<br>
+ name = m.group(1)<br>
+ if args.function is not None and name != args.function:<br>
+ # When filtering on a specific function, skip all others.<br>
+ continue<br>
+ is_in_function = is_in_function_start = True<br>
+<br>
+ if args.verbose:<br>
+ print>>sys.stderr, 'Writing %d fixed lines to %s...' % (<br>
+ len(fixed_lines), test)<br>
+ with open(test, 'w') as f:<br>
+ f.writelines([l + '\n' for l in fixed_lines])<br>
+<br>
+<br>
+if __name__ == '__main__':<br>
+ main()<br>
<br>
Propchange: llvm/trunk/utils/update_llc_test_checks.py<br>
------------------------------------------------------------------------------<br>
svn:executable = *<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br></div>