[flang-dev] Porting F18 tests to use LLVM lit

[Doerfert, Johannes via flang-dev]

Thanks for working on this!

On 01/15, Luke Ireland via flang-dev wrote:
> Further to the conversation in llvm-dev<http://lists.llvm.org/pipermail/llvm-dev/2020-January/138282.html>, regarding LLVM testing infrastructure/lit, and the successfully merged patch from Carol<https://github.com/flang-compiler/f18/pull/861/commits/1049aeb81e0195d3fa1050b21e93a3980a0b7245>, we’re continuing development on adding this infrastructure by porting the existing tests to be compatible with lit.
> 
> We’d like to propose the following approach:
> 
>   1.  Start off with a new directory, “test-lit” containing lit tests
>   2.  make check-all will run this and the old test directory together
>   3.  port over all old tests to this new directory as lit tests
>   4.  Once all tests are successfully ported, the old directory and test scripts are removed, and “test-lit” becomes “test”.
>   5.  All new tests will be created to be compatible with lit.
> During this transition phase, tests can either be written in the old style, and then ported with my script, or preferably, written to work with lit.

This sounds very reasonable.

> My PR has three patches:
> 
>   1.  A script which ports old style tests to new lit tests. At the moment it only handles semantic error tests, i.e. tests run with check_error.sh
>   2.  The output of the porting script on a subset of the tests for verification. The subset of tests should cover all variations of error tests in my script
>   3.  All the remaining error tests from semantics ported to lit.

Did you look at the llvm/utils/update_XXX_test_checks.py scripts? We
might want to add a scripts there if we can generate, not port, the
CHECK lines.

Cheers,
  Johannes

> My next steps:
> 
>   1.  Port remaining semantic test types
>      *   Symbol
>      *   Generic
>      *   Modfile
>   2.  Port decimal tests
>   3.  Port evaluate tests
>   4.  Port preprocessing tests (They aren’t in CMakeLists.txt, but are easily portable)
> Each step will come as a pull request in a similar format:
> 
>   1.  Script change to handle new test type
>   2.  Ported tests added
>      *   Either in 2 commits (verified, remaining – likely for changes with a large number of tests)
>      *   Or all in 1 commit (small number of tests)
>   3.  Old style tests removed
> My initial PR is here<https://github.com/flang-compiler/f18/pull/934>.
> 
> Thanks,
> Luke
> 
> Degree Apprentice,
> Arm Manchester

> _______________________________________________
> flang-dev mailing list
> flang-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev


-- 

Johannes Doerfert
Researcher

Argonne National Laboratory
Lemont, IL 60439, USA

jdoerfert at anl.gov
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/flang-dev/attachments/20200115/00d8341f/attachment-0001.sig>