[PATCH] Teach CMake build system to run lit's own test suite

[JF Bastien via llvm-commits]

On Sat, Jan 9, 2016 at 1:26 AM, Dan Liew <dan at su-root.co.uk> wrote:

> Hi JF,
>
> On 9 January 2016 at 01:16, JF Bastien <jfb at chromium.org> wrote:
> > Hi Dan,
> >
> > I don't know if I'm the only one getting this, but it seems like your
> patch
> > causes timeouts for me and for some bots I'm testing with (they're all
> > timing out after your commit).
>
> What is your timeout for your bots? If your system has the Python
> psutil module installed the lit timeout tests will run which are a
> little slow but **should not** hang indefinitely.
>

The current timeout is 20 minutes per step, and the entire bots used to
complete in less than 10 minutes (they do more than build and test LLVM).


```
> $ time ninja check-lit
> [2/2] Running lit's tests
> lit.py: lit.cfg:58: note: Found python psutil module
> Testing Time: 16.90s
>   Expected Passes    : 14
>
> real    0m17.096s
> user    0m13.610s
> sys     0m0.577s
> ```
>
>
>
> > I used the following to repro locally:
> >
> > cmake ../llvm -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DLLVM_BUILD_TESTS=ON
> > -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_ASSERTIONS=ON
> > -DLLVM_TARGETS_TO_BUILD="X86" && make check-all -j$(nproc)
> >
> > Things are stuck at "running all regression tests".
> >
> > I have the python processes at the bottom of this email hung (all
> disappear
> > when I kill make). I'm pretty suspicious of the infinite_loop.py test
> > considering things seem to loop forever :-)
>
> That test is deliberately running a program that loops forever to test
> that the ``--timeout=`` feature of lit works. If it hangs it likely
> means there is a bug with that feature.
>
> > The LLVM waterfall at http://lab.llvm.org:8011/console seems fine, so
> maybe
> > it's a Python version difference? The bots I'm using are pretty simple
> x86
> > machines, whereas my machine is a beefy one, so I don't think it's a
> problem
> > with too-slow or too-big machine.
>
> When I wrote the ``--timeout=`` tests I tested Python 2.7 and Python
> 3.4 with the version 3.2 of the Python psutil module
>
> Could you tell me
>
> - What OS you are using (e.g. Ubuntu 14.04LTS)
>

Ubuntu 14.04 LTS on 3.13.0-67-generic.

- What Python version you have
>

2.7.6


> - What version of the Python psutil module you have
>

1.2.1-1ubuntu2

With that I can have a go at reproducing your issue.
>
> > Would you have an idea of what's going on? I haven't dug into the code
> yet
> > since I figure you may already have a hunch.
>
> My hunch is that if you have an old version of psutil installed it
> might be behaving badly and failing to kill the processes (see
> ``killProcessAndChildren()`` in ``utils/lit/lit/util.py``).
> If that is the case this might accelerate my plans to remove the
> dependency on the psutil module.
>
> Note I'm seeing Windows buildbot failures (not related to lit timeout
> feature)
>
> http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/8760
> http://lab.llvm.org:8011/builders/clang-x86-win2008-selfhost/builds/5613
>
> so I need to revert my commit until I can work out why the tests are
> broken under Windows (eurgh...)
>
> Unfortunately it seems svn is down right now so I can't do anything
> until it's back up.
>
> Dan.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160109/08e3a42d/attachment.html>