[Mlir-commits] [mlir] Quantile Type and Low FP Support (PR #190321)

[llvmlistbot at llvm.org]

================
@@ -547,6 +548,111 @@ class CalibratedQuantizedType
   double getMax() const;
 };
 
+/*Syntax:
+
+    ```
+    quantile-type ::= `!quant.quantile` `<` type `:` type `,` `{` float-list `}`
+   `,` `<` int `,` int `>`? `>`
+    ```
+
+    A quantile type represents a quantile-based floating point encoding, where
+    discrete storage values are totally defined by the floating-point values
+   entries in a quantile lookup table of F8/F16/F32.
+
+    Optionally, explicit minimum and maximum storage values can be specified
+    after the LUT as `<min:max>`.
+
+    This type is used for weight compression schemes like NF4 (NormalizedFloat4)
+    and similar quantile-based formats.
+
+    Example:
+
+   MLIR:
+    !quant.quantile<ui4:f16, {-1.0,-0.696,0.0,0.079,1.0}>
+    !quant.quantile<ui4:f16, {-1.0,-0.696,0.0,0.079,1.0}, <-8,7>>
+*/
+
+class QuantileType
+    : public Type::TypeBase<QuantileType, QuantizedType,
+                            detail::QuantileTypeStorage,
+                            mlir::QuantStorageTypeInterface::Trait> {
+public:
+  using ImplType = detail::QuantileTypeStorage;
+  using Base::Base;
+
+  // Get the underlying type used for to store raw values.
+  Type getStorageType() const;
+
+  // Get primitive expressed type of data in quantiles.
+  // Note that we may convert FP8 data to FP16 for storage,
+  // but we should treat its expressed type as FP8 rather than FP16.
+  Type getQuantileType() const;
+
+  /// Return the quantile table of this float type.
----------------
vsimion26 wrote:

I think it is better to have support for fp64 maybe in the future there will be a usecase for it, I updated the description to match with this

https://github.com/llvm/llvm-project/pull/190321