[llvm-dev] distributed lit testing

[Christian Kühnel via llvm-dev]

+iwg at llvm.org <iwg at llvm.org>


On Wed, Feb 24, 2021 at 9:54 AM James Henderson via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi Victor,
>
> The lit test framework is the main testing framework used by LLVM. You can
> find the source code for it in the LLVM github repository (see in
> particular https://github.com/llvm/llvm-project/tree/main/llvm/utils/lit),
> and there is documentation available for it on the LLVM website -
> https://llvm.org/docs/TestingGuide.html gives the high-level picture of
> how LLVM is tested, whilst https://llvm.org/docs/CommandGuide/lit.htmlis
> more focused on lit specifically.
>
> Examples of where lit is used include the individual test files located in
> places like llvm/test, clang/test and lld/test within the github tree.
> These test directories include additional configuration files, some of
> which are configured when CMake is used to generate the build files for the
> LLVM project. If you aren't already familiar with LLVM, I highly recommend
> reading up on https://llvm.org/docs/GettingStarted.html, and following
> the steps to make sure you can build and run LLVM components locally.
>
> Lit works as a python process which spawns many child processes, each of
> which runs one or more of the tests located in the directory under test.
> These tests typically are a sequence of commands that use components of
> LLVM that have already been built. You can build the test dependencies and
> run the tests by building one of the CMake-generated targets called check-*
> (where * might be llvm, lld, clang, etc to run a test subset or "check-all"
> to run all known tests. Currently, the tests run in parallel on the user's
> machine, using the python multiprocessing library to do this. There also
> exists the --num-shards and related options which allows multiple computers
> to each run a subset of the tests. I am not too familiar on how this option
> is used in practice, but I believe it requires the computers to all have
> access to some shared filesystem which contains the tests and build
> artifacts, or to each have the same version checked out and to have been
> sent the full set of build artifacts to use. Others on this list might be
> able to clarify further.
>
> The project goal is to provide a framework for distributing these tests
> across multiple computers in a more flexible manner than the existing
> sharding mechanism. I can think of two different high-level options -
> either a layer on top of lit which uses the existing sharding mechanism
> somehow, or something built into the existing lit code that goes wide with
> the tests across the machines. It would be up to you to identify and
> implement a way forward doing this. The hope would be that this framework
> could be used for multiple different distributed systems, as described in
> the original project description on the Open Projects page.
>
> This project is intended to be a possible Google Summer of Code project.
> As such, to participate in it, you'd need to sign up on the GSOC website,
> and provide a project proposal there which details how you plan to solve
> the challenge. It would help your proposal get accepted if you can show
> some understanding of the lit testsuite, and some evidence of contributions
> to LLVM (perhaps in the form of additional testing you might identify that
> is missing in some tests, or by fixing one or more bugs from the LLVM
> bugzilla page, perhaps labelled with the "beginner" keyword). I am happy to
> work with you on your proposal if you are uncertain about anything, but the
> core of the proposal needs to come from you.
>
> I hope that gives you the information you are looking for. Please feel
> free to ask any further questions that you may have.
>
> James
>
> On Tue, 23 Feb 2021 at 17:28, Victor Kukshiev via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hello I am Victor Kukshiev (cetjs2 in IRC), 2rd course student of PetrSU
>> university.
>> Distributed lit testing idea is interested and possible for me, I think.
>> Could you tell us more about this project?
>> What is lit test suite?
>> I know python  language.
>> What do I participate in thiis project?
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210312/fe0b11f7/attachment.html>