<div dir="ltr">Hi Justin,<div><br></div><div>What I think we should do is key `use_clang_verify` on whether `cxx_under_test` is clang right away. Then for the time being we can add `// VERIFY` to the top of all</div><div>converted tests. Then we can choose to run it as a verify test if `use_clang_verify` is true and `// VERIFY` is present. Once we convert all the tests we can</div>
<div>remove the `// VERIFY` tags. </div><div><br></div><div>Thanks for getting this started.</div><div>/Eric</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Sep 2, 2014 at 10:55 PM, Justin Bogner <span dir="ltr"><<a href="mailto:mail@justinbogner.com" target="_blank">mail@justinbogner.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hey Eric,<br>
<br>
This implements the clang -verify idea that we discussed after r216317<br>
and converts one test as an example. WDYT?<br>
<span class="HOEnZb"><font color="#888888"><br>
-- Justin<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
Justin Bogner <<a href="mailto:mail@justinbogner.com">mail@justinbogner.com</a>> writes:<br>
> Author: bogner<br>
> Date: Tue Sep 2 23:32:08 2014<br>
> New Revision: 217009<br>
><br>
> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=217009&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=217009&view=rev</a><br>
> Log:<br>
> test: Allow using clang -verify for failures rather than exit 1<br>
><br>
> Currently, failure tests work by checking that compilation exits 1.<br>
> This can lead to tests that fail for the wrong reason, so it'd be<br>
> preferable to convert them to check for specific errors.<br>
><br>
> This adds use_clang_verify parameter that runs failure tests using<br>
> clang's -verify flag. I'll convert some tests in subsequent commits,<br>
> and once all of the tests are converted we should key this on whether<br>
> cxx_under_test is clang.<br>
><br>
> I've also converted one of the unique.ptr tests, since it's the one<br>
> that motivated the idea of using clang -verify when possible in the<br>
> review of r216317.<br>
><br>
> Modified:<br>
> libcxx/trunk/test/lit.cfg<br>
> libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/unique.ptr.runtime.ctor/default01.fail.cpp<br>
><br>
> Modified: libcxx/trunk/test/lit.cfg<br>
> URL:<br>
> <a href="http://llvm.org/viewvc/llvm-project/libcxx/trunk/test/lit.cfg?rev=217009&r1=217008&r2=217009&view=diff==============================================================================" target="_blank">http://llvm.org/viewvc/llvm-project/libcxx/trunk/test/lit.cfg?rev=217009&r1=217008&r2=217009&view=diff==============================================================================</a><br>
> --- libcxx/trunk/test/lit.cfg (original)<br>
> +++ libcxx/trunk/test/lit.cfg Tue Sep 2 23:32:08 2014<br>
> @@ -28,8 +28,10 @@ class LibcxxTestFormat(lit.formats.FileB<br>
> FOO.fail.cpp - Negative test case which is expected to fail compilation.<br>
> """<br>
><br>
> - def __init__(self, cxx_under_test, cpp_flags, ld_flags, exec_env):<br>
> + def __init__(self, cxx_under_test, use_verify_for_fail,<br>
> + cpp_flags, ld_flags, exec_env):<br>
> self.cxx_under_test = cxx_under_test<br>
> + self.use_verify_for_fail = use_verify_for_fail<br>
> self.cpp_flags = list(cpp_flags)<br>
> self.ld_flags = list(ld_flags)<br>
> self.exec_env = dict(exec_env)<br>
> @@ -116,13 +118,17 @@ class LibcxxTestFormat(lit.formats.FileB<br>
> if expected_compile_fail:<br>
> cmd = [self.cxx_under_test, '-c',<br>
> '-o', '/dev/null', source_path] + self.cpp_flags<br>
> - out, err, exitCode = self.execute_command(cmd)<br>
> - if exitCode == 1:<br>
> + expected_rc = 1<br>
> + if self.use_verify_for_fail:<br>
> + cmd += ['-Xclang', '-verify']<br>
> + expected_rc = 0<br>
> + out, err, rc = self.execute_command(cmd)<br>
> + if rc == expected_rc:<br>
> return lit.Test.PASS, ""<br>
> else:<br>
> report = """Command: %s\n""" % ' '.join(["'%s'" % a<br>
> for a in cmd])<br>
> - report += """Exit Code: %d\n""" % exitCode<br>
> + report += """Exit Code: %d\n""" % rc<br>
> if out:<br>
> report += """Standard Output:\n--\n%s--""" % out<br>
> if err:<br>
> @@ -190,6 +196,7 @@ class Configuration(object):<br>
> self.compile_flags = []<br>
> self.link_flags = []<br>
> self.use_system_lib = False<br>
> + self.use_clang_verify = False<br>
><br>
> if platform.system() not in ('Darwin', 'FreeBSD', 'Linux'):<br>
> self.lit_config.fatal("unrecognized system")<br>
> @@ -202,12 +209,24 @@ class Configuration(object):<br>
> val = default<br>
> return val<br>
><br>
> + def get_lit_bool(self, name):<br>
> + conf = self.get_lit_conf(name)<br>
> + if conf is None:<br>
> + return None<br>
> + if conf.lower() in ('1', 'true'):<br>
> + return True<br>
> + if conf.lower() in ('', '0', 'false'):<br>
> + return False<br>
> + self.lit_config.fatal(<br>
> + "parameter '{}' should be true or false".format(name))<br>
> +<br>
> def configure(self):<br>
> self.configure_cxx()<br>
> self.configure_triple()<br>
> self.configure_src_root()<br>
> self.configure_obj_root()<br>
> self.configure_use_system_lib()<br>
> + self.configure_use_clang_verify()<br>
> self.configure_env()<br>
> self.configure_std_flag()<br>
> self.configure_compile_flags()<br>
> @@ -218,6 +237,7 @@ class Configuration(object):<br>
> def get_test_format(self):<br>
> return LibcxxTestFormat(<br>
> self.cxx,<br>
> + self.use_clang_verify,<br>
> cpp_flags=['-nostdinc++'] + self.compile_flags,<br>
> ld_flags=['-nodefaultlibs'] + self.link_flags,<br>
> exec_env=self.env)<br>
> @@ -251,21 +271,22 @@ class Configuration(object):<br>
> # the locally built one; the former mode is useful for testing ABI<br>
> # compatibility between the current headers and a shipping dynamic<br>
> # library.<br>
> - use_system_lib_str = self.get_lit_conf('use_system_lib')<br>
> - if use_system_lib_str:<br>
> - if use_system_lib_str.lower() in ('1', 'true'):<br>
> - self.use_system_lib = True<br>
> - elif use_system_lib_str.lower() in ('', '0', 'false'):<br>
> - self.use_system_lib = False<br>
> - else:<br>
> - self.lit_config.fatal(<br>
> - 'user parameter use_system_lib should be 0 or 1')<br>
> - else:<br>
> + self.use_system_lib = self.get_lit_bool('use_system_lib')<br>
> + if self.use_system_lib is None:<br>
> # Default to testing against the locally built libc++ library.<br>
> self.use_system_lib = False<br>
> self.lit_config.note(<br>
> "inferred use_system_lib as: %r" % self.use_system_lib)<br>
><br>
> + def configure_use_clang_verify(self):<br>
> + '''If set, run clang with -verify on failing tests.'''<br>
> + self.use_clang_verify = self.get_lit_bool('use_clang_verify')<br>
> + if self.use_clang_verify is None:<br>
> + # TODO: Default this to True when using clang.<br>
> + self.use_clang_verify = False<br>
> + self.lit_config.note(<br>
> + "inferred use_clang_verify as: %r" % self.use_clang_verify)<br>
> +<br>
> def configure_features(self):<br>
> # Figure out which of the required locales we support<br>
> locales = {<br>
><br>
> Modified: libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/unique.ptr.runtime.ctor/default01.fail.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/unique.ptr.runtime.ctor/default01.fail.cpp?rev=217009&r1=217008&r2=217009&view=diff==============================================================================" target="_blank">http://llvm.org/viewvc/llvm-project/libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/unique.ptr.runtime.ctor/default01.fail.cpp?rev=217009&r1=217008&r2=217009&view=diff==============================================================================</a><br>
> --- libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/unique.ptr.runtime.ctor/default01.fail.cpp (original)<br>
> +++ libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/unique.ptr.runtime.ctor/default01.fail.cpp Tue Sep 2 23:32:08 2014<br>
> @@ -19,8 +19,9 @@<br>
><br>
> class Deleter<br>
> {<br>
> -<br>
> - Deleter() {}<br>
> + // expected-error@memory:* {{base class 'Deleter' has private default constructor}}<br>
> + // expected-note@memory:* + {{in instantiation of member function}}<br>
> + Deleter() {} // expected-note {{implicitly declared private here}}<br>
><br>
> public:<br>
><br>
><br>
><br>
> _______________________________________________<br>
> cfe-commits mailing list<br>
> <a href="mailto:cfe-commits@cs.uiuc.edu">cfe-commits@cs.uiuc.edu</a><br>
> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits</a><br>
</div></div></blockquote></div><br></div>