<div dir="ltr">Thanks for the review. Committed as r<span style="color:rgb(0,0,0)">211789.</span></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jun 26, 2014 at 6:37 PM, NAKAMURA Takumi <span dir="ltr"><<a href="mailto:geek4civic@gmail.com" target="_blank">geek4civic@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">LGTM.<br>
<br>
2014-06-27 1:36 GMT+09:00 Alexander Kornienko <<a href="mailto:alexfh@google.com">alexfh@google.com</a>>:<br>
<div class="HOEnZb"><div class="h5">> On Thu, Jun 26, 2014 at 6:09 PM, NAKAMURA Takumi <<a href="mailto:geek4civic@gmail.com">geek4civic@gmail.com</a>><br>
> wrote:<br>
>><br>
>> Since clang-tidy-diff.py is not a test script but a tool, it's fine to<br>
>> require modern version of python for it.<br>
>><br>
>> Note, your test script invokes python as "python", but lit might not<br>
>> run with "python".<br>
><br>
><br>
> Yep, this wasn't very reliable.<br>
><br>
>><br>
>> For example in CMake build, PYTHON_EXECUTABLE is used.<br>
>> Consider that it is out of $PATH, or it is like "python32".<br>
>><br>
>> Then you should import config.python_executable from llvm/test.<br>
><br>
><br>
> I'm don't know to import it (if you know, please enlighten me), but I know<br>
> how to generate from the <a href="http://lit.site.cfg.in" target="_blank">lit.site.cfg.in</a>. Here's the updated patch:<br>
><br>
> Index: test/Makefile<br>
> ===================================================================<br>
> --- test/Makefile (revision 211769)<br>
> +++ test/Makefile (working copy)<br>
> @@ -52,6 +52,7 @@<br>
> @$(ECHOPATH) s=@CLANG_TOOLS_SOURCE_DIR@=$(PROJ_SRC_DIR)/..=g >><br>
> lit.tmp<br>
> @$(ECHOPATH) s=@CLANG_TOOLS_BINARY_DIR@=$(PROJ_OBJ_DIR)/..=g >><br>
> lit.tmp<br>
> @$(ECHOPATH) s=@CLANG_TOOLS_DIR@=$(ToolDir)=g >> lit.tmp<br>
> + @$(ECHOPATH) s=@PYTHON_EXECUTABLE@=$(PYTHON)=g >> lit.tmp<br>
> @$(ECHOPATH) s=@TARGET_TRIPLE@=$(TARGET_TRIPLE)=g >> lit.tmp<br>
> @sed -f lit.tmp $(PROJ_SRC_DIR)/<a href="http://lit.site.cfg.in" target="_blank">lit.site.cfg.in</a> > $@<br>
> @-rm -f lit.tmp<br>
> Index: test/clang-tidy/clang-tidy-diff.cpp<br>
> ===================================================================<br>
> --- test/clang-tidy/clang-tidy-diff.cpp (revision 211769)<br>
> +++ test/clang-tidy/clang-tidy-diff.cpp (working copy)<br>
> @@ -1,6 +1,7 @@<br>
> +// REQUIRES: python27<br>
> // RUN: sed 's/placeholder_for_f/f/' %s > %t.cpp<br>
> // RUN: clang-tidy -checks=-*,misc-use-override %t.cpp -- -std=c++11 |<br>
> FileCheck -check-prefix=CHECK-SANITY %s<br>
> -// R_U_N: not diff -U0 %s %t.cpp | python<br>
> %S/../../clang-tidy/tool/clang-tidy-diff.py -checks=-*,misc-use-override --<br>
> -std=c++11 2>&1 | FileCheck %s<br>
> +// RUN: not diff -U0 %s %t.cpp | %python<br>
> %S/../../clang-tidy/tool/clang-tidy-diff.py -checks=-*,misc-use-override --<br>
> -std=c++11 2>&1 | FileCheck %s<br>
> struct A {<br>
> virtual void f() {}<br>
> virtual void g() {}<br>
> Index: test/lit.cfg<br>
> ===================================================================<br>
> --- test/lit.cfg (revision 211769)<br>
> +++ test/lit.cfg (working copy)<br>
> @@ -174,3 +174,10 @@<br>
> # ANSI escape sequences in non-dumb terminal<br>
> if platform.system() not in ['Windows']:<br>
> config.available_features.add('ansi-escape-sequences')<br>
> +<br>
> +config.substitutions.append( ('%python', config.python_executable) )<br>
> +<br>
> +import sys<br>
> +# Scripts using argparse need Python 2.7.<br>
> +if sys.version_info >= (2, 7):<br>
> + config.available_features.add('python27')<br>
> Index: test/<a href="http://lit.site.cfg.in" target="_blank">lit.site.cfg.in</a><br>
> ===================================================================<br>
> --- test/<a href="http://lit.site.cfg.in" target="_blank">lit.site.cfg.in</a> (revision 211769)<br>
> +++ test/<a href="http://lit.site.cfg.in" target="_blank">lit.site.cfg.in</a> (working copy)<br>
> @@ -7,6 +7,7 @@<br>
> config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"<br>
> config.clang_tools_binary_dir = "@CLANG_TOOLS_BINARY_DIR@"<br>
> config.clang_tools_dir = "@CLANG_TOOLS_DIR@"<br>
> +config.python_executable = "@PYTHON_EXECUTABLE@"<br>
> config.target_triple = "@TARGET_TRIPLE@"<br>
><br>
> # Support substitution of the tools and libs dirs with user parameters.<br>
> This is<br>
><br>
><br>
>><br>
>><br>
>> 2014-06-26 23:46 GMT+09:00 Alexander Kornienko <<a href="mailto:alexfh@google.com">alexfh@google.com</a>>:<br>
>> > It seems like we can't require Python 2.7 for running tests, but I'm<br>
>> > also<br>
>> > against rewriting the script using deprecated APIs. I'd better make this<br>
>> > test conditional on the availability of Python 2.7. What do you think?<br>
>> ><br>
>> > Here's the patch which implements this:<br>
>> ><br>
>> > Index: test/clang-tidy/clang-tidy-diff.cpp<br>
>> > ===================================================================<br>
>> > --- test/clang-tidy/clang-tidy-diff.cpp (revision 211769)<br>
>> > +++ test/clang-tidy/clang-tidy-diff.cpp (working copy)<br>
>> > @@ -1,6 +1,7 @@<br>
>> > +// REQUIRES: python27<br>
>> > // RUN: sed 's/placeholder_for_f/f/' %s > %t.cpp<br>
>> > // RUN: clang-tidy -checks=-*,misc-use-override %t.cpp -- -std=c++11 |<br>
>> > FileCheck -check-prefix=CHECK-SANITY %s<br>
>> > -// R_U_N: not diff -U0 %s %t.cpp | python<br>
>> > %S/../../clang-tidy/tool/clang-tidy-diff.py -checks=-*,misc-use-override<br>
>> > --<br>
>> > -std=c++11 2>&1 | FileCheck %s<br>
>> > +// RUN: not diff -U0 %s %t.cpp | python<br>
>> > %S/../../clang-tidy/tool/clang-tidy-diff.py -checks=-*,misc-use-override<br>
>> > --<br>
>> > -std=c++11 2>&1 | FileCheck %s<br>
>> > struct A {<br>
>> > virtual void f() {}<br>
>> > virtual void g() {}<br>
>> > Index: test/lit.cfg<br>
>> > ===================================================================<br>
>> > --- test/lit.cfg (revision 211769)<br>
>> > +++ test/lit.cfg (working copy)<br>
>> > @@ -174,3 +174,8 @@<br>
>> > # ANSI escape sequences in non-dumb terminal<br>
>> > if platform.system() not in ['Windows']:<br>
>> > config.available_features.add('ansi-escape-sequences')<br>
>> > +<br>
>> > +import sys<br>
>> > +# Scripts using argparse need Python 2.7.<br>
>> > +if sys.version_info >= (2, 7):<br>
>> > + config.available_features.add('python27')<br>
>> ><br>
>> ><br>
>> ><br>
>> > On Thu, Jun 26, 2014 at 1:45 AM, Alexander Kornienko <<a href="mailto:alexfh@google.com">alexfh@google.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> On Thu, Jun 26, 2014 at 1:24 AM, NAKAMURA Takumi <<a href="mailto:geek4civic@gmail.com">geek4civic@gmail.com</a>><br>
>> >> wrote:<br>
>> >>><br>
>> >>> 2014-06-25 23:09 GMT+09:00 Alexander Kornienko <<a href="mailto:alexfh@google.com">alexfh@google.com</a>>:<br>
>> >>> > Author: alexfh<br>
>> >>> > Date: Wed Jun 25 09:09:52 2014<br>
>> >>> > New Revision: 211698<br>
>> >>> ><br>
>> >>> > URL: <a href="http://llvm.org/viewvc/llvm-project?rev=211698&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=211698&view=rev</a><br>
>> >>> > Log:<br>
>> >>> > Add clang-tidy-diff.py script to run clang-tidy and display warnings<br>
>> >>> > on<br>
>> >>> > changed lines only.<br>
>> >>> ><br>
>> >>> > Reviewers: djasper<br>
>> >>> ><br>
>> >>> > Reviewed By: djasper<br>
>> >>> ><br>
>> >>> > Subscribers: cfe-commits<br>
>> >>> ><br>
>> >>> > Differential Revision: <a href="http://reviews.llvm.org/D4288" target="_blank">http://reviews.llvm.org/D4288</a><br>
>> >>> ><br>
>> >>> > Added:<br>
>> >>> > clang-tools-extra/trunk/clang-tidy/tool/clang-tidy-diff.py<br>
>> >>> > (with<br>
>> >>> > props)<br>
>> >>> > clang-tools-extra/trunk/test/clang-tidy/clang-tidy-diff.cpp<br>
>> >>> ><br>
>> >>> > Added: clang-tools-extra/trunk/clang-tidy/tool/clang-tidy-diff.py<br>
>> >>> > URL:<br>
>> >>> ><br>
>> >>> > <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clang-tidy/tool/clang-tidy-diff.py?rev=211698&view=auto" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clang-tidy/tool/clang-tidy-diff.py?rev=211698&view=auto</a><br>
>> >>> ><br>
>> >>> ><br>
>> >>> > ==============================================================================<br>
>> >>> > --- clang-tools-extra/trunk/clang-tidy/tool/clang-tidy-diff.py<br>
>> >>> > (added)<br>
>> >>> > +++ clang-tools-extra/trunk/clang-tidy/tool/clang-tidy-diff.py Wed<br>
>> >>> > Jun<br>
>> >>> > 25 09:09:52 2014<br>
>> >>> > @@ -0,0 +1,115 @@<br>
>> >>> > +#!/usr/bin/python<br>
>> >>> > +#<br>
>> >>> > +#===- clang-tidy-diff.py - ClangTidy Diff Checker ------------*-<br>
>> >>> > python -*--===#<br>
>> >>> > +#<br>
>> >>> > +# The LLVM Compiler Infrastructure<br>
>> >>> > +#<br>
>> >>> > +# This file is distributed under the University of Illinois Open<br>
>> >>> > Source<br>
>> >>> > +# License. See LICENSE.TXT for details.<br>
>> >>> > +#<br>
>> >>> ><br>
>> >>> ><br>
>> >>> > +#===------------------------------------------------------------------------===#<br>
>> >>> > +<br>
>> >>> > +r"""<br>
>> >>> > +ClangTidy Diff Checker<br>
>> >>> > +======================<br>
>> >>> > +<br>
>> >>> > +This script reads input from a unified diff, runs clang-tidy on all<br>
>> >>> > changed<br>
>> >>> > +files and outputs clang-tidy warnings in changed lines only. This<br>
>> >>> > is<br>
>> >>> > useful to<br>
>> >>> > +detect clang-tidy regressions in the lines touched by a specific<br>
>> >>> > patch.<br>
>> >>> > +Example usage for git/svn users:<br>
>> >>> > +<br>
>> >>> > + git diff -U0 HEAD^ | clang-tidy-diff.py -p1<br>
>> >>> > + svn diff --diff-cmd=diff -x-U0 | \<br>
>> >>> > + clang-tidy-diff.py -fix -checks=-*,misc-use-override<br>
>> >>> > +<br>
>> >>> > +"""<br>
>> >>> > +<br>
>> >>> > +import argparse<br>
>> >>><br>
>> >>> argparse has been introduced since 2.7. I suppose we can still use<br>
>> >>> 2.6.<br>
>> >><br>
>> >><br>
>> >> Python 2.7 has been available since 2010, and it should be available<br>
>> >> virtually everywhere. I've not seen any python-related buildbot<br>
>> >> failures<br>
>> >> after this patch, so I suppose they all use up-to-date python. Can you<br>
>> >> provide more details: where is python 2.6 still used and why do we need<br>
>> >> to<br>
>> >> support it?<br>
>> >><br>
>> >>><br>
>> >>> Suppressed in r211741. Could you rewrite to avoid argparse?<br>
>> >><br>
>> >><br>
>> >> Python 2.6 had optparse, which has been deprecated since 2.7. I don't<br>
>> >> think it's a good idea to rewrite the script using deprecated api.<br>
>> >><br>
>> >>><br>
>> >>> > +import json<br>
>> >>> > +import re<br>
>> >>> > +import subprocess<br>
>> >>> > +import sys<br>
>> >>> > +<br>
>> >>> > +<br>
>> >>> > +def main():<br>
>> >>> > + parser = argparse.ArgumentParser(description=<br>
>> >>> > + 'Reformat changed lines in diff.<br>
>> >>> > Without -i '<br>
>> >>> > + 'option just output the diff<br>
>> >>> > that<br>
>> >>> > would be '<br>
>> >>> > + 'introduced.')<br>
>> >>> > + parser.add_argument('-clang-tidy-binary', metavar='PATH',<br>
>> >>> > + default='clang-tidy',<br>
>> >>> > + help='path to clang-tidy binary')<br>
>> >>> > + parser.add_argument('-p', metavar='NUM', default=0,<br>
>> >>> > + help='strip the smallest prefix containing P<br>
>> >>> > slashes')<br>
>> >>> > + parser.add_argument('-regex', metavar='PATTERN', default=None,<br>
>> >>> > + help='custom pattern selecting file paths to<br>
>> >>> > reformat '<br>
>> >>> > + '(case sensitive, overrides -iregex)')<br>
>> >>> > + parser.add_argument('-iregex', metavar='PATTERN', default=<br>
>> >>> > +<br>
>> >>> > r'.*\.(cpp|cc|c\+\+|cxx|c|cl|h|hpp|m|mm|inc)',<br>
>> >>> > + help='custom pattern selecting file paths to<br>
>> >>> > reformat '<br>
>> >>> > + '(case insensitive, overridden by -regex)')<br>
>> >>> > +<br>
>> >>> > + parser.add_argument('-fix', action='store_true', default=False,<br>
>> >>> > + help='apply suggested fixes')<br>
>> >>> > + parser.add_argument('-checks',<br>
>> >>> > + help='checks filter, when not specified, use<br>
>> >>> > clang-tidy '<br>
>> >>> > + 'default',<br>
>> >>> > + default='')<br>
>> >>> > + clang_tidy_args = []<br>
>> >>> > + argv = sys.argv[1:]<br>
>> >>> > + if '--' in argv:<br>
>> >>> > + clang_tidy_args.extend(argv[argv.index('--'):])<br>
>> >>> > + argv = argv[:argv.index('--')]<br>
>> >>> > +<br>
>> >>> > + args = parser.parse_args(argv)<br>
>> >>> > +<br>
>> >>> > + # Extract changed lines for each file.<br>
>> >>> > + filename = None<br>
>> >>> > + lines_by_file = {}<br>
>> >>> > + for line in sys.stdin:<br>
>> >>> > + match = re.search('^\+\+\+\ (.*?/){%s}(\S*)' % args.p, line)<br>
>> >>> > + if match:<br>
>> >>> > + filename = match.group(2)<br>
>> >>> > + if filename == None:<br>
>> >>> > + continue<br>
>> >>> > +<br>
>> >>> > + if args.regex is not None:<br>
>> >>> > + if not re.match('^%s$' % args.regex, filename):<br>
>> >>> > + continue<br>
>> >>> > + else:<br>
>> >>> > + if not re.match('^%s$' % args.iregex, filename,<br>
>> >>> > re.IGNORECASE):<br>
>> >>> > + continue<br>
>> >>> > +<br>
>> >>> > + match = re.search('^@@.*\+(\d+)(,(\d+))?', line)<br>
>> >>> > + if match:<br>
>> >>> > + start_line = int(match.group(1))<br>
>> >>> > + line_count = 1<br>
>> >>> > + if match.group(3):<br>
>> >>> > + line_count = int(match.group(3))<br>
>> >>> > + if line_count == 0:<br>
>> >>> > + continue<br>
>> >>> > + end_line = start_line + line_count - 1;<br>
>> >>> > + lines_by_file.setdefault(filename, []).append([start_line,<br>
>> >>> > end_line])<br>
>> >>> > +<br>
>> >>> > + if len(lines_by_file) == 0:<br>
>> >>> > + print "No relevant changes found."<br>
>> >>> > + sys.exit(0)<br>
>> >>> > +<br>
>> >>> > + line_filter_json = json.dumps(<br>
>> >>> > + [{"name" : name, "lines" : lines_by_file[name]} for name in<br>
>> >>> > lines_by_file],<br>
>> >>> > + separators = (',', ':'))<br>
>> >>> > +<br>
>> >>> > + # Run clang-tidy on files containing changes.<br>
>> >>> > + command = [args.clang_tidy_binary]<br>
>> >>> > + command.append('-line-filter=\'' + line_filter_json + '\'')<br>
>> >>> > + if args.fix:<br>
>> >>> > + command.append('-fix')<br>
>> >>> > + if args.checks != '':<br>
>> >>> > + command.append('-checks=\'' + args.checks + '\'')<br>
>> >>> > + command.extend(lines_by_file.keys())<br>
>> >>> > + command.extend(clang_tidy_args)<br>
>> >>> > +<br>
>> >>> > + sys.exit(subprocess.call(' '.join(command), shell=True))<br>
>> >>> > +<br>
>> >>> > +if __name__ == '__main__':<br>
>> >>> > + main()<br>
>> >>> ><br>
>> >>> > Propchange:<br>
>> >>> > clang-tools-extra/trunk/clang-tidy/tool/clang-tidy-diff.py<br>
>> >>> ><br>
>> >>> ><br>
>> >>> > ------------------------------------------------------------------------------<br>
>> >>> > svn:executable = *<br>
>> >>> ><br>
>> >>> > Added: clang-tools-extra/trunk/test/clang-tidy/clang-tidy-diff.cpp<br>
>> >>> > URL:<br>
>> >>> ><br>
>> >>> > <a href="http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clang-tidy/clang-tidy-diff.cpp?rev=211698&view=auto" target="_blank">http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/clang-tidy/clang-tidy-diff.cpp?rev=211698&view=auto</a><br>
>> >>> ><br>
>> >>> ><br>
>> >>> > ==============================================================================<br>
>> >>> > --- clang-tools-extra/trunk/test/clang-tidy/clang-tidy-diff.cpp<br>
>> >>> > (added)<br>
>> >>> > +++ clang-tools-extra/trunk/test/clang-tidy/clang-tidy-diff.cpp Wed<br>
>> >>> > Jun<br>
>> >>> > 25 09:09:52 2014<br>
>> >>> > @@ -0,0 +1,18 @@<br>
>> >>> > +// RUN: sed 's/placeholder_for_f/f/' %s > %t.cpp<br>
>> >>> > +// RUN: clang-tidy -checks=-*,misc-use-override %t.cpp --<br>
>> >>> > -std=c++11 |<br>
>> >>> > FileCheck -check-prefix=CHECK-SANITY %s<br>
>> >>> > +// RUN: not diff -u0 %s %t.cpp | python<br>
>> >>> > %S/../../clang-tidy/tool/clang-tidy-diff.py<br>
>> >>> > -checks=-*,misc-use-override --<br>
>> >>> > -std=c++11 2>&1 | FileCheck %s<br>
>> >>> > +struct A {<br>
>> >>> > + virtual void f() {}<br>
>> >>> > + virtual void g() {}<br>
>> >>> > +};<br>
>> >>> > +// CHECK-NOT: warning<br>
>> >>> > +struct B : public A {<br>
>> >>> > + void placeholder_for_f() {}<br>
>> >>> > +// CHECK-SANITY: [[@LINE-1]]:8: warning: Use exactly<br>
>> >>> > +// CHECK: [[@LINE-2]]:8: warning: Use exactly<br>
>> >>> > + void g() {}<br>
>> >>> > +// CHECK-SANITY: [[@LINE-1]]:8: warning: Use exactly<br>
>> >>> > +// CHECK-NOT: warning:<br>
>> >>> > +};<br>
>> >>> > +// CHECK-SANITY-NOT: Suppressed<br>
>> >>> > +// CHECK: Suppressed 1 warnings (1 due to line filter).<br>
>> ><br>
>> ><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div><div><font color="#666666"><span style="border-top-width:2px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;border-top-style:solid;border-right-style:solid;border-bottom-style:solid;border-left-style:solid;border-top-color:rgb(213,15,37);border-right-color:rgb(213,15,37);border-bottom-color:rgb(213,15,37);border-left-color:rgb(213,15,37);padding-top:2px;margin-top:2px">Alexander Kornienko |</span><span style="border-top-width:2px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;border-top-style:solid;border-right-style:solid;border-bottom-style:solid;border-left-style:solid;border-top-color:rgb(51,105,232);border-right-color:rgb(51,105,232);border-bottom-color:rgb(51,105,232);border-left-color:rgb(51,105,232);padding-top:2px;margin-top:2px"> Software Engineer |</span></font><span style="border-top-width:2px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;border-top-style:solid;border-right-style:solid;border-bottom-style:solid;border-left-style:solid;border-top-color:rgb(0,153,57);border-right-color:rgb(0,153,57);border-bottom-color:rgb(0,153,57);border-left-color:rgb(0,153,57);padding-top:2px;margin-top:2px"><font color="#666666"> </font><a href="mailto:alexfh@google.com" style="color:rgb(17,85,204)" target="_blank">alexfh@google.com</a> |</span><span style="border-top-width:2px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;border-top-style:solid;border-right-style:solid;border-bottom-style:solid;border-left-style:solid;border-top-color:rgb(238,178,17);border-right-color:rgb(238,178,17);border-bottom-color:rgb(238,178,17);border-left-color:rgb(238,178,17);padding-top:2px;margin-top:2px"> Google Germany, Munich</span></div>
</div></div>
</div>