[all-commits] [llvm/llvm-project] 1824e4: [MLIR, Python] Support converting boolean numpy arr...

[Kasper Nielsen via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 1824e45cd799a19fb9b5f9a84f9a0197157af8c8
      https://github.com/llvm/llvm-project/commit/1824e45cd799a19fb9b5f9a84f9a0197157af8c8
  Author: Kasper Nielsen <kasper0406 at gmail.com>
  Date:   2024-11-13 (Wed, 13 Nov 2024)

  Changed paths:
    M mlir/lib/Bindings/Python/IRAttributes.cpp
    M mlir/test/python/ir/array_attributes.py

  Log Message:
  -----------
  [MLIR,Python] Support converting boolean numpy arrays to and from mlir attributes (unrevert) (#115481)

This PR re-introduces the functionality of
https://github.com/llvm/llvm-project/pull/113064, which was reverted in
https://github.com/llvm/llvm-project/commit/0a68171b3c67503f7143856580f1b22a93ef566e
due to memory lifetime issues.

Notice that I was not able to re-produce the ASan results myself, so I
have not been able to verify that this PR really fixes the issue.

---

Currently it is unsupported to:
1. Convert a MlirAttribute with type i1 to a numpy array
2. Convert a boolean numpy array to a MlirAttribute

Currently the entire Python application violently crashes with a quite
poor error message https://github.com/pybind/pybind11/issues/3336

The complication handling these conversions, is that MlirAttribute
represent booleans as a bit-packed i1 type, whereas numpy represents
booleans as a byte array with 8 bit used per boolean.

This PR proposes the following approach:
1. When converting a i1 typed MlirAttribute to a numpy array, we can not
directly use the underlying raw data backing the MlirAttribute as a
buffer to Python, as done for other types. Instead, a copy of the data
is generated using numpy's unpackbits function, and the result is send
back to Python.
2. When constructing a MlirAttribute from a numpy array, first the
python data is read as a uint8_t to get it converted to the endianess
used internally in mlir. Then the booleans are bitpacked using numpy's
bitpack function, and the bitpacked array is saved as the MlirAttribute
representation.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications