[libcxx] r217009 - test: Allow using clang -verify for failures rather than exit 1
Justin Bogner
mail at justinbogner.com
Tue Sep 2 23:24:07 PDT 2014
Eric Fiselier <eric at efcs.ca> writes:
> Hi Justin,
>
> What I think we should do is key `use_clang_verify` on whether
> `cxx_under_test ` is clang right away. Then for the time being we can
> add `// VERIFY` to the top of all converted tests. Then we can choose
> to run it as a verify test if `use_clang_verify` is true and `//
> VERIFY` is present. Once we convert all the tests we can remove the
> `// VERIFY` tags.
This is a great idea. I've implemented the gist of it in r217017, but I
haven't actually added the clang detection yet. If you're testing the
just built clang, setting `--param=use_clang_verify=true` will make the
tests marked with `// USE_VERIFY` use verify now.
It's not clear to me what the best way to detect clang is, since (1) we
have to do it in lit to handle cxx_under_test being set, and (2) we need
to make sure that we have the appropriate features of -verify in the
host clang (notably r212735, which I used in my example conversion but
first appears in 3.5)
> Thanks for getting this started.
> /Eric
>
> On Tue, Sep 2, 2014 at 10:55 PM, Justin Bogner <mail at justinbogner.com> wrote:
>
> Hey Eric,
>
> This implements the clang -verify idea that we discussed after r216317
> and converts one test as an example. WDYT?
>
> -- Justin
>
> Justin Bogner <mail at justinbogner.com> writes:
> > Author: bogner
> > Date: Tue Sep 2 23:32:08 2014
> > New Revision: 217009
> >
> > URL: http://llvm.org/viewvc/llvm-project?rev=217009&view=rev
> > Log:
> > test: Allow using clang -verify for failures rather than exit 1
> >
> > Currently, failure tests work by checking that compilation exits 1.
> > This can lead to tests that fail for the wrong reason, so it'd be
> > preferable to convert them to check for specific errors.
> >
> > This adds use_clang_verify parameter that runs failure tests using
> > clang's -verify flag. I'll convert some tests in subsequent commits,
> > and once all of the tests are converted we should key this on whether
> > cxx_under_test is clang.
> >
> > I've also converted one of the unique.ptr tests, since it's the one
> > that motivated the idea of using clang -verify when possible in the
> > review of r216317.
> >
> > Modified:
> > libcxx/trunk/test/lit.cfg
> > libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/
> unique.ptr.runtime.ctor/default01.fail.cpp
> >
> > Modified: libcxx/trunk/test/lit.cfg
> > URL:
> > http://llvm.org/viewvc/llvm-project/libcxx/trunk/test/lit.cfg?rev=217009
> &r1=217008&r2=217009&view=diff============================================
> ==================================
> > --- libcxx/trunk/test/lit.cfg (original)
> > +++ libcxx/trunk/test/lit.cfg Tue Sep 2 23:32:08 2014
> > @@ -28,8 +28,10 @@ class LibcxxTestFormat(lit.formats.FileB
> > FOO.fail.cpp - Negative test case which is expected to fail
> compilation.
> > """
> >
> > - def __init__(self, cxx_under_test, cpp_flags, ld_flags, exec_env):
> > + def __init__(self, cxx_under_test, use_verify_for_fail,
> > + cpp_flags, ld_flags, exec_env):
> > self.cxx_under_test = cxx_under_test
> > + self.use_verify_for_fail = use_verify_for_fail
> > self.cpp_flags = list(cpp_flags)
> > self.ld_flags = list(ld_flags)
> > self.exec_env = dict(exec_env)
> > @@ -116,13 +118,17 @@ class LibcxxTestFormat(lit.formats.FileB
> > if expected_compile_fail:
> > cmd = [self.cxx_under_test, '-c',
> > '-o', '/dev/null', source_path] + self.cpp_flags
> > - out, err, exitCode = self.execute_command(cmd)
> > - if exitCode == 1:
> > + expected_rc = 1
> > + if self.use_verify_for_fail:
> > + cmd += ['-Xclang', '-verify']
> > + expected_rc = 0
> > + out, err, rc = self.execute_command(cmd)
> > + if rc == expected_rc:
> > return lit.Test.PASS, ""
> > else:
> > report = """Command: %s\n""" % ' '.join(["'%s'" % a
> > for a in cmd])
> > - report += """Exit Code: %d\n""" % exitCode
> > + report += """Exit Code: %d\n""" % rc
> > if out:
> > report += """Standard Output:\n--\n%s--""" % out
> > if err:
> > @@ -190,6 +196,7 @@ class Configuration(object):
> > self.compile_flags = []
> > self.link_flags = []
> > self.use_system_lib = False
> > + self.use_clang_verify = False
> >
> > if platform.system() not in ('Darwin', 'FreeBSD', 'Linux'):
> > self.lit_config.fatal("unrecognized system")
> > @@ -202,12 +209,24 @@ class Configuration(object):
> > val = default
> > return val
> >
> > + def get_lit_bool(self, name):
> > + conf = self.get_lit_conf(name)
> > + if conf is None:
> > + return None
> > + if conf.lower() in ('1', 'true'):
> > + return True
> > + if conf.lower() in ('', '0', 'false'):
> > + return False
> > + self.lit_config.fatal(
> > + "parameter '{}' should be true or false".format(name))
> > +
> > def configure(self):
> > self.configure_cxx()
> > self.configure_triple()
> > self.configure_src_root()
> > self.configure_obj_root()
> > self.configure_use_system_lib()
> > + self.configure_use_clang_verify()
> > self.configure_env()
> > self.configure_std_flag()
> > self.configure_compile_flags()
> > @@ -218,6 +237,7 @@ class Configuration(object):
> > def get_test_format(self):
> > return LibcxxTestFormat(
> > self.cxx,
> > + self.use_clang_verify,
> > cpp_flags=['-nostdinc++'] + self.compile_flags,
> > ld_flags=['-nodefaultlibs'] + self.link_flags,
> > exec_env=self.env)
> > @@ -251,21 +271,22 @@ class Configuration(object):
> > # the locally built one; the former mode is useful for testing
> ABI
> > # compatibility between the current headers and a shipping
> dynamic
> > # library.
> > - use_system_lib_str = self.get_lit_conf('use_system_lib')
> > - if use_system_lib_str:
> > - if use_system_lib_str.lower() in ('1', 'true'):
> > - self.use_system_lib = True
> > - elif use_system_lib_str.lower() in ('', '0', 'false'):
> > - self.use_system_lib = False
> > - else:
> > - self.lit_config.fatal(
> > - 'user parameter use_system_lib should be 0 or 1')
> > - else:
> > + self.use_system_lib = self.get_lit_bool('use_system_lib')
> > + if self.use_system_lib is None:
> > # Default to testing against the locally built libc++
> library.
> > self.use_system_lib = False
> > self.lit_config.note(
> > "inferred use_system_lib as: %r" % self.use_system_lib)
> >
> > + def configure_use_clang_verify(self):
> > + '''If set, run clang with -verify on failing tests.'''
> > + self.use_clang_verify = self.get_lit_bool('use_clang_verify')
> > + if self.use_clang_verify is None:
> > + # TODO: Default this to True when using clang.
> > + self.use_clang_verify = False
> > + self.lit_config.note(
> > + "inferred use_clang_verify as: %r" %
> self.use_clang_verify)
> > +
> > def configure_features(self):
> > # Figure out which of the required locales we support
> > locales = {
> >
> > Modified: libcxx/trunk/test/utilities/memory/unique.ptr/
> unique.ptr.runtime/unique.ptr.runtime.ctor/default01.fail.cpp
> > URL: http://llvm.org/viewvc/llvm-project/libcxx/trunk/test/utilities/
> memory/unique.ptr/unique.ptr.runtime/unique.ptr.runtime.ctor/
> default01.fail.cpp?rev=217009&r1=217008&r2=217009&view=diff===============
> ===============================================================
> > --- libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/
> unique.ptr.runtime.ctor/default01.fail.cpp (original)
> > +++ libcxx/trunk/test/utilities/memory/unique.ptr/unique.ptr.runtime/
> unique.ptr.runtime.ctor/default01.fail.cpp Tue Sep 2 23:32:08 2014
> > @@ -19,8 +19,9 @@
> >
> > class Deleter
> > {
> > -
> > - Deleter() {}
> > + // expected-error at memory:* {{base class 'Deleter' has private
> default constructor}}
> > + // expected-note at memory:* + {{in instantiation of member function}}
> > + Deleter() {} // expected-note {{implicitly declared private here}}
> >
> > public:
> >
> >
> >
> > _______________________________________________
> > cfe-commits mailing list
> > cfe-commits at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
More information about the cfe-commits
mailing list