[llvm-dev] RFC Adding Fortran tests to LLVM Test Suite

[Renato Golin via llvm-dev]

On Wed, 25 Nov 2020 at 16:08, Johannes Doerfert <johannesdoerfert at gmail.com>
wrote:

> FWIW, adding any Fortran benchmark will only make sense while adding
> also test suite capabilities. So the first step is compiling Fortran
> via the test suite, but the first real tests/benchmarks were supposed
> to be the SPEC ones because they are the easiest to add, it literally
> just takes a modification of the existing CMAke file (I hope ;) ).
>

Not everyone can run SPEC, regardless of how easy it is to integrate it
with the test-suite.

If you only add infrastructure to build Fortran programs inside SPEC, then
your change would be biased towards an external benchmark that is private
to some companies.

To make sure that the infrastructure you create for building Fortran
programs across all areas of the test-suite, you need to "show your work"
to the upstream community first by making it work for everyone.

Public build-bots will start building those tests and benchmarks (remember,
it's not just benchmarks in there), and you'll need some time to adjust
strategy until it all works across the board.

Then adding your side would be a straightforward step. But adding it as a
first thing, could mean other parts will take a lot longer to come, or to
be broken upstream and disabled, or never to come.

We had a similar situation in the test-suite for a long time regarding
cross-builds. There was some qemu support for a while that in the end only
worked in a convoluted way for some downstream project.

We don't want that to be the case for Fortran. If Fortran is
truly upstream, then its benchmarks and (at the very least) tests need to
be upstream as well. SPEC is "in addition" to that, because it's private.

If you want support from the community, you have to give something that is
beneficial to the community, however important your part is to your project.

Including some micro benchmarks with the Fortran plumbing is probably
> a good idea to make sure "it works". Then we'll work on adding benchmarks
> as described.


I fear "it works" will just end up "it works for SPEC and a few dummy
examples", which will bitrot like other examples in the past.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201125/3d91f6b0/attachment.html>