[flang-dev] [EXTERNAL] flang test questions

[McCormick, Pat via flang-dev]

On Jul 21, 2021, at 5:26 PM, Damian Rouson <rouson at lbl.gov<mailto:rouson at lbl.gov>> wrote:

For testing, I was originally planning to use the Vegetables<https://gitlab.com/everythingfunctional/vegetables> unit testing framework, which I like because it supports writing tests in the form of a specification.  I would also like to add a capability for building the tests using the Fortran Package Manager (fpm<https://github.com/fortran-lang/fpm>).  Doing so is trivial -- much simpler than modifying the CMake and with the benefit that it's trivial to set things up so that fpm automatically downloads any test dependencies such as Vegetables and we don't have to touch the CMake scripts.  Alternatively, if this proves problematic in any way, I'll commit a simple assertion utility borrowed from the Sourcery<https://github.com/sourceryinstitute/sourcery> library.

Hi Damian,

Just a quick note to give you a feel for the testing infrastructure here — it is much broader than what it might appear at first glance as you have to look beyond Flang.

The testing mechanisms need to be usable/stable across the community as testing and releases are done LLVM-wide (across all projects: Clang, MLIR, Flang, LLVM, etc.).  As a result, any new external package dependencies could have an impact across established mechanisms across all projects (e.g., see https://lab.llvm.org/buildbot for a glance at coverage, platforms, etc.).

For this reason, I would suggest adopting existing capabilities from the current mechanisms used across the LLVM infrastructure as your starting point.  A key step will be to make sure any approach taken can be run without additional requirements on the broader community — or we will have to open a discussion with the broader community about expanding the supported infrastructure.  Like everything, there are always pros and cons, but keeping Flang's approach usable and sound as a component of the larger code base is paramount to maintaining LLVM's established testing and release process.

Thanks,

—Pat

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/flang-dev/attachments/20210726/ec7069f6/attachment.html>