[all-commits] [llvm/llvm-project] 0677e5: [mlir][python] Allow contexts to be created with a...

[Stella Laurenzo via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 0677e54653e593ee90bb747fd75605f0bed47137
      https://github.com/llvm/llvm-project/commit/0677e54653e593ee90bb747fd75605f0bed47137
  Author: Stella Laurenzo <stellaraccident at gmail.com>
  Date:   2023-11-11 (Sat, 11 Nov 2023)

  Changed paths:
    M mlir/lib/Bindings/Python/IRCore.cpp
    M mlir/python/mlir/_mlir_libs/__init__.py

  Log Message:
  -----------
  [mlir][python] Allow contexts to be created with a custom thread pool. (#72042)

The existing initialization sequence always enables multi-threading at
MLIRContext construction time, making it impractical to provide a
customized thread pool.

Here, this is changed to always create the context with threading
disabled, process all site-specific init hooks (which can set thread
pools) and ultimately enable multi-threading unless if site-configured
to not do so.

This should preserve the existing user-visible initialization behavior
while also letting downstreams ensure that contexts are always created
with a shared thread pool. This was tested with IREE, which has such a
concept. Using site-specific thread tuning produced up to 2x single
compilation job behavior and customization of batch compilation (i.e. as
part of a build system) to utilize half the memory and run the entire
test suite ~2x faster. Given this, I believe that the additional
configurability can well pay for itself for implementations that use it.
We may also want to present user-level Python APIs for controlling
threading configuration in the future.