[flang-dev] Porting F18 tests to use LLVM lit

[Stephen Scalpone via flang-dev]

Hi Luke,

During the transition, when writing a lit test, are there any special guidelines beyond choosing a distinct test name?

After the final step of removing old tests, will you rename the test-lit directory to test?

Thanks for working on this!

 - Steve

On 1/15/20, 10:59 AM, "flang-dev on behalf of Doerfert, Johannes via flang-dev" <flang-dev-bounces at lists.llvm.org on behalf of flang-dev at lists.llvm.org> wrote:

    External email: Use caution opening links or attachments
    
    
    Thanks for working on this!
    
    On 01/15, Luke Ireland via flang-dev wrote:
    > Further to the conversation in llvm-dev<http://lists.llvm.org/pipermail/llvm-dev/2020-January/138282.html>, regarding LLVM testing infrastructure/lit, and the successfully merged patch from Carol<https://github.com/flang-compiler/f18/pull/861/commits/1049aeb81e0195d3fa1050b21e93a3980a0b7245>, we’re continuing development on adding this infrastructure by porting the existing tests to be compatible with lit.
    >
    > We’d like to propose the following approach:
    >
    >   1.  Start off with a new directory, “test-lit” containing lit tests
    >   2.  make check-all will run this and the old test directory together
    >   3.  port over all old tests to this new directory as lit tests
    >   4.  Once all tests are successfully ported, the old directory and test scripts are removed, and “test-lit” becomes “test”.
    >   5.  All new tests will be created to be compatible with lit.
    > During this transition phase, tests can either be written in the old style, and then ported with my script, or preferably, written to work with lit.
    
    This sounds very reasonable.
    
    > My PR has three patches:
    >
    >   1.  A script which ports old style tests to new lit tests. At the moment it only handles semantic error tests, i.e. tests run with check_error.sh
    >   2.  The output of the porting script on a subset of the tests for verification. The subset of tests should cover all variations of error tests in my script
    >   3.  All the remaining error tests from semantics ported to lit.
    
    Did you look at the llvm/utils/update_XXX_test_checks.py scripts? We
    might want to add a scripts there if we can generate, not port, the
    CHECK lines.
    
    Cheers,
      Johannes
    
    > My next steps:
    >
    >   1.  Port remaining semantic test types
    >      *   Symbol
    >      *   Generic
    >      *   Modfile
    >   2.  Port decimal tests
    >   3.  Port evaluate tests
    >   4.  Port preprocessing tests (They aren’t in CMakeLists.txt, but are easily portable)
    > Each step will come as a pull request in a similar format:
    >
    >   1.  Script change to handle new test type
    >   2.  Ported tests added
    >      *   Either in 2 commits (verified, remaining – likely for changes with a large number of tests)
    >      *   Or all in 1 commit (small number of tests)
    >   3.  Old style tests removed
    > My initial PR is here<https://github.com/flang-compiler/f18/pull/934>.
    >
    > Thanks,
    > Luke
    >
    > Degree Apprentice,
    > Arm Manchester
    
    > _______________________________________________
    > flang-dev mailing list
    > flang-dev at lists.llvm.org
    > https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev
    
    
    --
    
    Johannes Doerfert
    Researcher
    
    Argonne National Laboratory
    Lemont, IL 60439, USA
    
    jdoerfert at anl.gov
    


-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------