[PATCH] D147493: [ELF] Cap parallel::strategy to 8 cores when --threads= is unspecified

Wed Apr 5 11:04:00 PDT 2023

andrewng added a comment.

I've re-run my example but this time I'm using the Windows process times (instead of Python script timer) and cycle counts. Also removing the output file between each link. Similar to before these are mean values for 16 iterations of a link. I've included 1 thread and that's now the "baseline".

| **Threads** | **Time (s)** | **Time gain (%)** | **Cycles (M)** | **Cycles overhead (%)** |
| 1           | 6.903        | -                 | 26,947         | -                       |
| 4           | 3.028        | 56.14             | 35,170         | 30.51                   |
| 8           | 2.405        | 65.16             | 46,298         | 71.81                   |
| 12          | 2.289        | 66.84             | 57,053         | 111.72                  |
| 16          | 2.240        | 67.54             | 72,185         | 167.87                  |
| 20          | 2.270        | 67.13             | 83,375         | 209.40                  |
| 24          | 2.278        | 67.01             | 93,264         | 246.10                  |
|

The results are pretty similar, with diminishing wall time improvements up to 16 threads (although at this point the system will be leaning more on SMT threads). As expected, these wall time improvements come at a significant cost in cycles. So as I've already mentioned, it really comes down to what's most important, e.g. minimum time, efficiency or some balance between the two. This is also somewhat dependent on the nature of the build, i.e. a build that will perform many links compared to a build that only does just one. It also depends on the parallelism of the build system itself. This problem has been around for sometime but it's becoming more prominent due to the higher core/thread counts that are now available. I don't think there's really a simple solution that "fits all".

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D147493/new/

https://reviews.llvm.org/D147493