[clang] [include-mapping] Python fixes (PR #65401)

Cassie Jones via cfe-commits cfe-commits at lists.llvm.org
Tue Sep 5 12:25:59 PDT 2023


https://github.com/porglezomp created https://github.com/llvm/llvm-project/pull/65401:

I had to regenerate the include mapping while resolving a downstream merge conflict, and ran into two issue on my machine. These shouldn't change anything, just make things work on my config.

- Move the `multiprocessing.Pool` initializer to a top-level function, it was previously causing a pickle failure with my machine's python, specifically complaining about having to pickle the lambda or a local function. Works fine with a top-level function.
- Change the `env python` to `env python3` for convenience. The interpreter is installed as python3 on macOS.

It looks like about 2/3 of the scripts in `clang/tools/` are using `env python3` over `env python` so the latter seems a reasonable change to make.

>From b8cac30c34d7c26dd3ec8e3e00678d2034e9f51c Mon Sep 17 00:00:00 2001
From: Cassie Jones <cassie_jones at apple.com>
Date: Tue, 5 Sep 2023 12:12:45 -0700
Subject: [PATCH] [include-mapping] Python fixes

- Move the multiprocessing.Pool initializer to a top-level function, it
  was previously causing a pickle failure with my machine's python.
- Change the `env python` to `env python3` for convenience
---
 clang/tools/include-mapping/cppreference_parser.py | 8 +++++---
 clang/tools/include-mapping/gen_std.py             | 2 +-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/clang/tools/include-mapping/cppreference_parser.py b/clang/tools/include-mapping/cppreference_parser.py
index cefdbeaf334e1c4..c1a1f9a86d05fc3 100644
--- a/clang/tools/include-mapping/cppreference_parser.py
+++ b/clang/tools/include-mapping/cppreference_parser.py
@@ -176,6 +176,10 @@ def _GetSymbols(pool, root_dir, index_page_name, namespace, variants_to_accept):
     return symbols
 
 
+def signal_ignore_initializer():
+    return signal.signal(signal.SIGINT, signal.SIG_IGN)
+
+
 def GetSymbols(parse_pages):
     """Get all symbols by parsing the given pages.
 
@@ -192,9 +196,7 @@ def GetSymbols(parse_pages):
     symbols = []
     # Run many workers to process individual symbol pages under the symbol index.
     # Don't allow workers to capture Ctrl-C.
-    pool = multiprocessing.Pool(
-        initializer=lambda: signal.signal(signal.SIGINT, signal.SIG_IGN)
-    )
+    pool = multiprocessing.Pool(initializer=signal_ignore_initializer)
     try:
         for root_dir, page_name, namespace in parse_pages:
             symbols.extend(
diff --git a/clang/tools/include-mapping/gen_std.py b/clang/tools/include-mapping/gen_std.py
index 57a5a6772ba894a..fcd3bd0d843ea16 100755
--- a/clang/tools/include-mapping/gen_std.py
+++ b/clang/tools/include-mapping/gen_std.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # ===- gen_std.py -  ------------------------------------------*- python -*--===#
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.



More information about the cfe-commits mailing list