[lldb-dev] Too many open files

[Todd Fiala via lldb-dev]

(swapped out the lldb list for the newer one)

On Fri, Oct 2, 2015 at 5:37 PM, Todd Fiala <todd.fiala at gmail.com> wrote:

> Hmm, sounds suspicious.
>
> Can you try running the tests with two options and see if you get
> different results?
>
> # should be equivalent for the default on Windows, thus should match your
> above results.  This one uses a thread per worker queue.
> --test-runner-name threading-pool
>
> # should use a different test runner.  This one uses a process per worker
> queue.
> --test-runner-name multiprocessing-pool
>
> Aside from that, it seems like the total number of open files is exceeding
> some process/system maximum, which sounds like (maybe) we're leaking files
> somewhere.  Not enough info yet to guess where that might be coming in
> from, but maybe a part of the test runner isn't closing files somewhere.
>
> The other thing you can try is reducing the total number of threads, with:
> --threads {some-number-lower-than-your-total-number-of-logical-cores}
>
> in the event that your machine has a mongo number of logical cores, and
> perhaps it is trying to do too much.  (In that case, the
> multiprocessing-pool runner might actually help).
>
> Thanks!
>
> -Todd
>
> On Fri, Oct 2, 2015 at 5:31 PM, Adrian McCarthy <amccarth at google.com>
> wrote:
>
>> When running LLDB tests on Windows, I started getting a "too many open
>> files" error from Python.  I used git bisect to narrow it down to this
>> revision:
>>
>> http://llvm.org/viewvc/llvm-project?view=revision&revision=249182
>>
>> The error output is:
>>
>> Command invoked: D:\src\Python-2.7.9\PCbuild\python_d.exe
>> D:\src\llvm\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
>> D:/src/llvm/build/ninja/bin/lldb.exe -s
>> D:/src/llvm/build/ninja/lldb-test-traces -u CXXFLAGS -u CFLAGS
>> --enable-crash-dialog -C D:\src\llvm\build\ninja_release\bin\clang.exe
>> --inferior -p TestRecursiveTypes.py D:\src\llvm\llvm\tools\lldb\test
>> --event-add-entries worker_index=7:int
>> 384 out of 400 test suites processed - TestRecursiveTypes.py
>>       Traceback (most recent call last):
>>   File "D:/src/llvm/llvm/tools/lldb/test/dotest.py", line 1457, in
>> <module>
>>   File "D:\src\llvm\llvm\tools\lldb\test\dosep.py", line 1355, in main
>>   File "D:\src\llvm\llvm\tools\lldb\test\dosep.py", line 968, in
>> walk_and_invoke
>>   File "D:\src\llvm\llvm\tools\lldb\test\dosep.py", line 1095, in <lambda>
>>   File "D:\src\llvm\llvm\tools\lldb\test\dosep.py", line 889, in
>> threading_test_runner_pool
>>   File "D:\src\llvm\llvm\tools\lldb\test\dosep.py", line 774, in
>> map_async_run_loop
>>   File "D:\src\Python-2.7.9\Lib\multiprocessing\pool.py", line 558, in get
>> OSError: [Errno 24] Too many open files
>> [77809 refs]
>> ninja: build stopped: subcommand failed.
>>
>>
>> Any clue what might have caused this or what can be done to fix it?
>>
>> It's Friday afternoon, so there's no urgency from my perspective.  I'll
>> probably get back to this on Monday morning.
>>
>> Thanks,
>> Adrian McCarthy
>>
>
>
>
> --
> -Todd
>



-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151002/a478ffc7/attachment.html>