[llvm-dev] lld and thread over-subscription

[Hal Finkel via llvm-dev]

Hi Rui, et al.,

I was experimenting yesterday with building lld on my POWER7 PPC64/Linux machine, and ran into an unfortunate problem. When running the regressions tests under lit, almost all of the tests fail like this:

terminate called after throwing an instance of 'std::system_error'
  what():  Resource temporarily unavailable
...
5  libc.so.6       0x00000080b7847238 abort + 4293480680
6  libstdc++.so.6  0x00000fff94f0f004 __gnu_cxx::__verbose_terminate_handler() + 4294099316
7  libstdc++.so.6  0x00000fff94f0bc84
8  libstdc++.so.6  0x00000fff94f0bccc std::terminate() + 4294087956
9  libstdc++.so.6  0x00000fff94f0c0c4 __cxa_throw + 4294088780
10 libstdc++.so.6  0x00000fff94f816e0 std::__throw_system_error(int) + 4294526808
11 libstdc++.so.6  0x00000fff94f83d30 std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>) + 4294534936
12 lld             0x000000001002a278
...

which seems to indicate a core problem here with dealing with thread-resource exhaustion. For almost all tests, running them individually (or using lit -j 1) works without a problem. We could deal with this by limiting the number of threads lld uses when running regression tests, or limit the number of threads that lit uses when running lld tests (as we currently do with the OpenMP runtime tests), but I'm somewhat concerned that users will run into this program regardless with heavily-parallelized builds.

We could try to catch exceptions that otherwise come from ThreadPoolExecutor's constructor, but do we compile with exceptions enabled?

Thanks again,
Hal

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory