[llvm-dev] lld and thread over-subscription

[Rui Ueyama via llvm-dev]

On Thu, Oct 1, 2015 at 9:35 AM, Hal Finkel <hfinkel at anl.gov> wrote:

> Hi Rui, et al.,
>
> I was experimenting yesterday with building lld on my POWER7 PPC64/Linux
> machine, and ran into an unfortunate problem. When running the regressions
> tests under lit, almost all of the tests fail like this:
>
> terminate called after throwing an instance of 'std::system_error'
>   what():  Resource temporarily unavailable
> ...
> 5  libc.so.6       0x00000080b7847238 abort + 4293480680
> 6  libstdc++.so.6  0x00000fff94f0f004
> __gnu_cxx::__verbose_terminate_handler() + 4294099316
> 7  libstdc++.so.6  0x00000fff94f0bc84
> 8  libstdc++.so.6  0x00000fff94f0bccc std::terminate() + 4294087956
> 9  libstdc++.so.6  0x00000fff94f0c0c4 __cxa_throw + 4294088780
> 10 libstdc++.so.6  0x00000fff94f816e0 std::__throw_system_error(int) +
> 4294526808
> 11 libstdc++.so.6  0x00000fff94f83d30
> std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) +
> 4294534936
> 12 lld             0x000000001002a278
> ...
>
> which seems to indicate a core problem here with dealing with
> thread-resource exhaustion. For almost all tests, running them individually
> (or using lit -j 1) works without a problem. We could deal with this by
> limiting the number of threads lld uses when running regression tests, or
> limit the number of threads that lit uses when running lld tests (as we
> currently do with the OpenMP runtime tests), but I'm somewhat concerned
> that users will run into this program regardless with heavily-parallelized
> builds.
>
> We could try to catch exceptions that otherwise come from
> ThreadPoolExecutor's constructor, but do we compile with exceptions enabled?
>

I guess we do not want to enable exceptions to deal with the issue. Are
COFF tests failing, or just ELF tests? If ELF tests for the old LLD are
failing, the best way would be to not use threads in the old LLD. It has
lingering threading issues.


> Thanks again,
> Hal
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151001/82d0b486/attachment.html>