[llvm-branch-commits] [Support] parallelFor: enable caller participation (PR #209690)

[Fangrui Song via llvm-branch-commits]

https://github.com/MaskRay created https://github.com/llvm/llvm-project/pull/209690

`parallelFor` dispatches its work to the thread pool and blocks the calling
thread in the `TaskGroup` destructor until the workers finish. The caller
thus stays idle instead of joining the work as modern parallel libraries do,
and `numactl -C 0-7 ld.lld --threads=8` ends up slower than
`numactl -C 0-8 ld.lld --threads=8`.

Run one worker on the calling thread instead. This requires that no
`parallelFor` body reads `getThreadIndex()`, which is unset on the caller;
the last such user is removed by #209687.

In the jobserver mode: only pool workers acquire job slots, so a
jobserver-limited process may briefly exceed its granted concurrency by
one thread. This is acceptable, better than pessimizing the common
non-jobserver case, and parallelFor is unused by jobserver users.