[llvm-dev] RFC Adding Fortran tests to LLVM Test Suite

[Renato Golin via llvm-dev]

On Tue, 24 Nov 2020 at 20:16, Romero, Nichols A. via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> In support of ongoing efforts with the new Flang compiler that was
> recently added to LLVM Project, we plan to expand the LLVM Test Suite to
> include additional Fortran tests. This will require some infrastructure
> work, e.g., to specify a Fortran compiler and flags which will then enable
> the Fortran tests.
>

Hi Nick,

This sounds like a no-brainer to me (ie. obviously good thing to have).

We are focusing on tests in the following area:
>
> - "smaller" language-centric tests,
>

We already have some for C and C++, it would be nice to reuse the same
infrastructure (or make it better while you're at it). It's a good proof of
concept to the rest of the infrastructure that building Fortran programs
works in the test-suite. I'd start here and only add those in the first
patch.

- high-performance computing proxy-apps (particularly from Department of
> Energy projects), similar to the C/C++ proxy apps we already have,
>

We had some trouble with the results of those programs in the past, when
checking them (ie. diff) against the "golden standard". You should be able
to change the driver program (the main function) to output some values or
aggregations that don't change the precision of the calculations on
different architectures and OSs, but still can detect differences if the
numbers change too much. It's a fine tuning process that takes a while but
makes for robust checks.

Some programs used to compute a hash of the output and compare that, but it
was really hard to find out what's wrong by just looking at hashes. It also
doesn't help with natural floating point variations between targets, so
best to avoid those.

- OpenMP tests: multi-threaded and GPU offload (once the OpenMP/parallelism
> testing support was merged into the LLVM Test Suite, separate thread).
>

Sounds great! Two birds. :)

As a first step we'll include the necessary CMake glue for the Fortran SPEC
> benchmarks, similar to CMake files we use to run the C/C++ ones.
>

I imagined the first step to make it work on the test-suite itself, not on
the SPEC link. I believe these are two different objectives. One is to make
the test-suite validate Fortran code generation at a higher level than just
the Lit tests and make sure the upstream code is on par by having public
buildbots on it, while the other is to improve downstream testing on
non-public benchmarks that will not have a direct effect on commits and
general progress validation.

Both are important, but the upstream part is more important for the
upstream community. I'd favour that to gain community support and do the
SPEC part on the side.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201125/5d97688a/attachment-0001.html>