[llvm-dev] [Openmp-dev] [cfe-dev] RFC: End-to-end testing
David Greene via llvm-dev
llvm-dev at lists.llvm.org
Wed Oct 16 12:54:24 PDT 2019
Renato Golin via Openmp-dev <openmp-dev at lists.llvm.org> writes:
> But if we have some consensus on doing a clean job, then I would
> actually like to have that kind of intermediary check (diagnostics,
> warnings, etc) on most test-suite tests, which would cover at least
> the main vectorisation issues. Later, we could add more analysis
> tools, if we want.
I think this makes a lot of sense.
> It would be as simple as adding CHECK lines on the execution of the
> compilation process (in CMake? Make? wrapper?) and keep the check
> files with the tests / per file.
> I think we're on the same page regarding almost everything, but
> perhaps I haven't been clear enough on the main point, which I think
> it's pretty simple. :)
Personally, I still find source-to-asm tests to be highly valuable and I
don't think we need test-suite for that. Such tests don't (usually)
depend on system libraries (headers may occasionally be an issue but I
would argue that the test is too fragile in that case).
So maybe we separate concerns. Use test-suite to do the kind of
system-level testing you've discussed but still allow some tests in a
monorepo top-level directory that test across components but don't
depend on system configurations.
If people really object to a top-level monorepo test directory I guess
they could go into test-suite but that makes it much more cumbersome to
run what really should be very simple tests.
More information about the llvm-dev