[llvm-dev] RFC Adding Fortran tests to LLVM Test Suite

[Romero, Nichols A. via llvm-dev]

Renato, See replies below.

From: Renato Golin <rengolin at gmail.com>
Date: Sunday, December 6, 2020 at 6:22 AM
To: Nichols Romero <naromero at anl.gov>
Cc: Johannes Doerfert <johannesdoerfert at gmail.com>, via llvm-dev <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] RFC Adding Fortran tests to LLVM Test Suite

On Sat, 5 Dec 2020 at 06:15, Romero, Nichols A. <naromero at anl.gov<mailto:naromero at anl.gov>> wrote:
As I was planning this out in my head, I do think in the first differential we would add the CMake plumbing and a simple opens source program or two (e.g. hand-coded GEMM or something along those lines). SPEC would come in the next batch.

That sounds like a good initial plan. Do you also plan to add more OSS applications, benchmarks and tests right after SPEC?

Yes, I am working on identifying some OSS applications and benchmarks.


The main reason for the test-suite is not benchmark, but end-to-end regression testing on multiple architectures. We can only claim to support languages and targets if we can compile entire applications, run them and get the expected results.

So having a set of language tests and some real world applications on the test-suite will be required for us to claim Fortran support of any kind.

Agreed.

Those are also your best friends in making sure all the work you've done on the front-end doesn't regress on any target, from release to release, or during normal development.

That's why I'm pushing to have those tests and applications as soon as possible. It's for Flang's own benefit.

On a related note, are there any buildbots running the llvm-test-suite or are folks just running llvm-test-suite manually?

Plenty, on both testing and benchmark modes. It's also part of the release process on all architectures.

Are they in some public dashboard like this one? http://lab.llvm.org:8011/#/console


You should add support for both testing and benchmark modes, so that the benchmarks (not SPEC) get regression tested as well as be able to report and compare performance numbers.

The goal is to have an analogous set-up to what is done now for C/C++. I am just getting up to speed now on it.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20201209/3bf6c160/attachment-0001.html>