[PATCH] D157884: [AIX] Limit `extract_symbols.py`'s Pools Size to 8

[Qiongsi Wu via Phabricator via llvm-commits]

qiongsiwu1 created this revision.
qiongsiwu1 added reviewers: daltenty, john.brawn, thieta.
Herald added a project: All.
qiongsiwu1 requested review of this revision.
Herald added subscribers: llvm-commits, wangpc.
Herald added a project: LLVM.

On AIX, `extract_symbols.py`'s default pool size may be too big for the logical partition(LPAR)'s allocated memory, and may lead to intermittent IO errors. This patch limits the pool size to 8 to avoid such errors.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D157884

Files:
  llvm/utils/extract_symbols.py


Index: llvm/utils/extract_symbols.py
===================================================================
--- llvm/utils/extract_symbols.py
+++ llvm/utils/extract_symbols.py
@@ -22,6 +22,7 @@
 import subprocess
 import multiprocessing
 import argparse
+import platform
 
 # Define a function which extracts a list of pairs of (symbols, is_def) from a
 # library using llvm-nm becuase it can work both with regular and bitcode files.
@@ -428,7 +429,12 @@
     # Extract symbols from libraries in parallel. This is a huge time saver when
     # doing a debug build, as there are hundreds of thousands of symbols in each
     # library.
-    pool = multiprocessing.Pool()
+    # FIXME: On AIX, the default pool size can be too big for a logical
+    #        partition's allocated memory, and can lead to an out of memory
+    #        IO error. We are setting the pool size to 8 to avoid such
+    #        errors at the moment, and will look for a graceful solution later.
+    pool = multiprocessing.Pool(8) if platform.system() == "AIX" \
+                                   else multiprocessing.Pool()
     try:
         # Only one argument can be passed to the mapping function, and we can't
         # use a lambda or local function definition as that doesn't work on


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D157884.549949.patch
Type: text/x-patch
Size: 1271 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230814/c67f75e0/attachment.bin>