[llvm] 3bcce9c - [AIX] Limit `extract_symbols.py`'s Pools Size to 8

[Qiongsi Wu via llvm-commits]

Author: Qiongsi Wu
Date: 2023-08-21T09:48:12-04:00
New Revision: 3bcce9ca224c1f1a940f19beef76fba491fceb18

URL: https://github.com/llvm/llvm-project/commit/3bcce9ca224c1f1a940f19beef76fba491fceb18
DIFF: https://github.com/llvm/llvm-project/commit/3bcce9ca224c1f1a940f19beef76fba491fceb18.diff

LOG: [AIX] Limit `extract_symbols.py`'s Pools Size to 8

On AIX, `extract_symbols.py`'s default pool size may be too big for the logical partition(LPAR)'s allocated memory, and may lead to intermittent IO errors. This patch limits the pool size to 8 to avoid such errors.

Reviewed By: daltenty, thieta

Differential Revision: https://reviews.llvm.org/D157884

Added: 
    

Modified: 
    llvm/utils/extract_symbols.py

Removed: 
    


################################################################################
diff  --git a/llvm/utils/extract_symbols.py b/llvm/utils/extract_symbols.py
index 9238828d7ce856..10fdf14acd1586 100755
--- a/llvm/utils/extract_symbols.py
+++ b/llvm/utils/extract_symbols.py
@@ -22,6 +22,7 @@
 import subprocess
 import multiprocessing
 import argparse
+import platform
 
 # Define a function which extracts a list of pairs of (symbols, is_def) from a
 # library using llvm-nm becuase it can work both with regular and bitcode files.
@@ -428,7 +429,12 @@ def parse_tool_path(parser, tool, val):
     # Extract symbols from libraries in parallel. This is a huge time saver when
     # doing a debug build, as there are hundreds of thousands of symbols in each
     # library.
-    pool = multiprocessing.Pool()
+    # FIXME: On AIX, the default pool size can be too big for a logical
+    #        partition's allocated memory, and can lead to an out of memory
+    #        IO error. We are setting the pool size to 8 to avoid such
+    #        errors at the moment, and will look for a graceful solution later.
+    pool = multiprocessing.Pool(8) if platform.system() == "AIX" \
+                                   else multiprocessing.Pool()
     try:
         # Only one argument can be passed to the mapping function, and we can't
         # use a lambda or local function definition as that doesn't work on