[lldb-dev] [cfe-dev] [Openmp-dev] [llvm-dev] RFC: End-to-end testing

David Blaikie via lldb-dev lldb-dev at lists.llvm.org
Wed Oct 16 12:58:05 PDT 2019

On Wed, Oct 16, 2019 at 12:54 PM David Greene via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> Renato Golin via Openmp-dev <openmp-dev at lists.llvm.org> writes:
> > But if we have some consensus on doing a clean job, then I would
> > actually like to have that kind of intermediary check (diagnostics,
> > warnings, etc) on most test-suite tests, which would cover at least
> > the main vectorisation issues. Later, we could add more analysis
> > tools, if we want.
> I think this makes a lot of sense.
> > It would be as simple as adding CHECK lines on the execution of the
> > compilation process (in CMake? Make? wrapper?) and keep the check
> > files with the tests / per file.
> Yep.
> > I think we're on the same page regarding almost everything, but
> > perhaps I haven't been clear enough on the main point, which I think
> > it's pretty simple. :)
> Personally, I still find source-to-asm tests to be highly valuable and I
> don't think we need test-suite for that.  Such tests don't (usually)
> depend on system libraries (headers may occasionally be an issue but I
> would argue that the test is too fragile in that case).
> So maybe we separate concerns.  Use test-suite to do the kind of
> system-level testing you've discussed but still allow some tests in a
> monorepo top-level directory that test across components but don't
> depend on system configurations.

I'm inclined to the direction suggested by others that the monorepo is
orthogonal to this issue and top level tests might not be the right thing.

lldb already does end-to-end testing in its tests, for instance.

Clang does in some tests (the place I always hit is anything that's
configured API-wise on the MCContext - there's no way to test that
configuration on the clang boundary, so the only test that we can write is
one that tests the effect of that API/programmatic configuration done by
clang to the MCContext (function sections, for instance) - in some cases
I've just skipped the testing, in others I've written the end-to-end test
in clang (& an LLVM test for the functionality that uses llvm-mc or

> If people really object to a top-level monorepo test directory I guess
> they could go into test-suite but that makes it much more cumbersome to
> run what really should be very simple tests.
>                    -David
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20191016/bf3bded8/attachment.html>

More information about the lldb-dev mailing list