[llvm-dev] RFC: Add bitcode tests to test-suite

[Alina Sbirlea via llvm-dev]

Hi all,

TL;DR: Add *.bc to test-suite; llc *.bc; run some.

We would like to propose adding bitcode tests to the llvm test-suite.

Recent LLVM bugs [2-4] prompted us to look into upstreaming a subset of the
tests the Halide library [1] is running and we'd like the community's
feedback on moving forward with this.

Halide uses LLVM and can generate bitcode, but we cannot add C++ tests to
test-suite without including the library itself.
This proposal is also potentially useful for other cases where there is no
C++ front-end.

As a first step we are interested in adding a set of correctness tests, for
testing the IR without running the tests. Since these tests are generated,
they are not instrumented like the .ll files in trunk, however we believe
checking that llc runs without errors is still useful.
The bitcode files for Halide may also be large, so including them as
regression tests is not an option. If the smaller tests are found to be
valuable or covering cases no other tests cover, we can instrument them and
move them into the llvm trunk further along, but that is not the goal of
this proposal.
In addition, we're not sure whether the format for the tests should be .ll
or .bc, we're open to either.

After this first step, we're interested in upstreaming bitcode tests and
also running them.
We are very interested in tests for multiple architectures, aarch64 in
particular, since this is where we have seen things break. This may
motivate adding .ll files rather than .bc in order to include the "RUN:"
target.
Where would these tests reside and with what directory structure? (similar
to test/CodeGen?)

Suggestion on what's the best approach for extending the test-suite
framework for this proposal are more than welcome.

This is just the high-level overview to start off the discussion, I'm sure
there are many more aspects to touch on. Looking forward to your feedback!

Thanks,
Alina

[1] http://halide-lang.org/
[2] Broken: r259800 => Fixed: r260131
[3] Broken: r260569 => Fixed: r260701
[4] https://llvm.org/bugs/show_bug.cgi?id=26642
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160217/d8e48b0e/attachment.html>