[flang-commits] [flang] [flang] Add FLANG_PARALLEL_COMPILE_JOBS option (PR #127364)

[Michael Kruse via flang-commits]

Meinersbur wrote:

Considering global user settings is the least we can do. In addition to LLVM_RAM_PER_COMPILE_JOB there is also LLVM_PARALLEL_COMPILE_JOBS. If user-friendlyness is the goal, we should predefine LLVM_RAM_PER_COMPILE_JOB for known compilers. This concrete PR adds just another trap for new users.

However, with the current support for pools, there is a trade-off to make:

1. Two different pools for flang and other (compile) jobs
   a.  LLVM_PARALLEL_COMPILE_JOBS * LLVM_RAM_PER_COMPILE_JOB + FLANG_PARALLEL_COMPILE_JOBS * FLANG_RAM_PER_COMPILE_JOB <= SYSTEM_RAM: Wasted parallelism while no flang jobs are running
   b.  Otherwise: Risk of swapping/OOM
3. Sharing the same pool for all jobs (e.g. FLANG_RAM_PER_COMPILE_JOB  = 4 * LLVM_RAM_PER_COMPILE_JOB)
   a. LLVM_PARALLEL_COMPILE_JOBS * FLANG_RAM_PER_COMPILE_JOB <= SYSTEM_RAM: Wasted parallelism when few jobs actually use FLANG_RAM_PER_COMPILE_JOB 
   b. Otherwise: Risk of swapping/OOM

Neither of them really looks better than the other.

https://github.com/llvm/llvm-project/pull/127364