[cfe-dev] Positive tests in the clang test suite?

[Nemanja Ivanovic via cfe-dev]

Also Joel,

If you'd like some documentation on testing for LLVM, you can refer to
http://llvm.org/docs/TestingGuide.html.

But of course, David's summary gives you an excellent overview.

On Thu, Jan 18, 2018 at 5:48 PM, David Chisnall via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

>
> > On 18 Jan 2018, at 15:46, Joel Jacobson via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
> >
> > Dear Clang hackers,
> >
> > I'm trying to understand how Clang is tested,
> > I've looked at lots of .c and .cpp files in ./test/,
> > and I see a lot of "// expected-warning" and "// expected-error"
> annotations,
> > but I don't see any positive tests checking for expected values?
>
> These are the expected values.  You are looking at Sema checks, which
> check that the semantic analyser is correctly diagnosing errors in programs.
>
> >
> > Other compilers such as rustc uses macros such as assert_eq, e.g.:
> > #[test]
> > fn test_extend_specialization() {
> >    let mut a = BinaryHeap::from(vec![-10, 1, 2, 3, 3]);
> >    let b = BinaryHeap::from(vec![-20, 5, 43]);
> >
> >    a.extend(b);
> >
> >    assert_eq!(a.into_sorted_vec(), [-20, -10, 1, 2, 3, 3, 5, 43]);
> > }
>
> This looks like an executable test (i.e. one where the output of the
> compiler must be run).  The clang test suite does not contain these, all of
> the CodeGen tests check that the correct IR is generated, but it’s then up
> to LLVM to correctly generate code from this.  In LLVM’s test suite, you
> will find tests that check that the expected assembly is generated from IR
> and that the expected binary sequences are generated from assembly.
>
> The advantage of a test suite of this structure is that it can run on any
> platform (there is no requirement, for example, that the platform that is
> currently running clang can run code using Windows or DWARF exceptions to
> test both of those).  The tests are all unit tests and so you can quickly
> (typically in under 30 seconds) run just the relevant part of the test
> suite while debugging a new feature.
>
> The down side is that they are not integration tests and so don’t check
> that the code is correct, only that it is the same as last time.  The LLVM
> Nightly Test suite contains a number of executable tests, but these
> typically take much longer to run and so are run by the CI infrastructure
> on a variety of different platforms.
>
> David
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180118/8a67795a/attachment.html>