[Mlir-commits] [mlir] 47347d2 - Quantile Type and Low FP Support (#190321)
llvmlistbot at llvm.org
llvmlistbot at llvm.org
Tue Jun 2 07:14:49 PDT 2026
Author: vsimion26
Date: 2026-06-02T15:14:42+01:00
New Revision: 47347d24f0a9f68ae60a54ce2ab46a023b739b8d
URL: https://github.com/llvm/llvm-project/commit/47347d24f0a9f68ae60a54ce2ab46a023b739b8d
DIFF: https://github.com/llvm/llvm-project/commit/47347d24f0a9f68ae60a54ce2ab46a023b739b8d.diff
LOG: Quantile Type and Low FP Support (#190321)
# **QuantileType: Composing with Interface-Based Storage Types in MLIR
Quantization**
## **Context**
Recent [community
work](https://discourse.llvm.org/t/rfc-extending-uniformquantizedtype-with-interface-based-support-for-new-storage-types-in-quant-dialect/87803)
(RFC by Roman-Pevnyi, Aug 2025)
successfully extended UniformQuantizedType with a StorageTypeInterface.
This
made the quantization framework extensible, allowing new storage types
(Integer, Float8E5M2, Float8E4M3FN, NF4) to be plugged in without
modifying
core quantization logic.
## **Building on that work**
QuantileType follows the same interface-driven philosophy but addresses
a
different level of abstraction: the storage type itself.
The observation: Many low-precision storage types (ui4, si8, f8, NF4)
can be enhanced with a quantile lookup table. Rather than creating a new
complete quantized type for each variant (QuantileQuantizedType,
QuantileQuantizedPerAxisType, etc.), we insert an abstraction layer.
QuantileType is a builtin that wraps any storage type with quantile
metadata:
quantile<ui4:f16, {-1.0, -0.696, ..., 1.0}>
quantile<si8:f32, {-2.0, -1.0, 0.0, 1.0, 2.0}>
quantile<f8E4M3FN:f16, {...}>
This storage abstraction then composes naturally with existing
quantization:
!quant.uniform<quantile<ui4:f16, {...}>:f32, scale:zeropoint>
## **Architecture**
Roman's StorageTypeInterface allows UniformQuantizedType to work with
any
compliant storage type. QuantileType extends this by making it possible
to
augment any storage type WITH quantile information, creating composable
layers:
Builtin QuantileType (unified storage + quantiles)
↓ (implements StorageTypeInterface)
UniformQuantizedType (uniform quantization logic)
↓
Hardware-specific lowering
## **Key benefits**
1. SINGLE INTERFACE FOR ALL QUANTILE SCHEMES
With QuantileType: One parameterized abstraction
quantile<ui4:f16, {nf4_table}>
quantile<ui4:f16, {custom_table}>
All compose with !quant.uniform naturally
[NF4
Type](https://github.com/openvinotoolkit/npu_compiler/blob/90b6098b9ee96055d633dc520354434bae22e336/src/vpux_compiler/include/vpux/compiler/core/types/quantile_float/types.hpp#L62)
& [NF4
Table](https://github.com/openvinotoolkit/npu_compiler/blob/90b6098b9ee96055d633dc520354434bae22e336/src/vpux_compiler/src/core/types/quantile_float/types.cpp#L116-L132)
2. EXTENSIBILITY FOR NEW STORAGE TYPES
When a new low-fp storage type is added (FP3, FPx, etc.),
it automatically works with quantiles:
quantile<fpx:f16, {...}>
No new dialect types needed. The type just implements
StorageTypeInterface
and QuantileType wraps it.
3. CLEAN SEPARATION OF CONCERNS
StorageTypeInterface:
- Defines what storage types must provide (width, signedness, min/max
values)
QuantileType (builtin abstraction):
- Wraps any StorageTypeInterface-compliant type with a lookup table
- Acts as a "storage + quantile mapping" layer
UniformQuantizedType (quantization logic):
- Works with any StorageTypeInterface, including QuantileType
## **Example IR**
In uniform quantization context:
!elem_type = !quant.uniform<quantile<ui4:f16, {-1.0, ..., 1.0}>:f32,
0.01:128>
QuantileType is a natural extension of the StorageTypeInterface
architecture. It:
- Provides a unified abstraction for quantile-enhanced storage
- Avoids type explosion by parameterizing rather than creating variants
- Maintains clean separation between storage concerns and quantization
logic
- Enables portability and reusability across MLIR consumers
Added:
mlir/test/Dialect/Quant/invalid-quantile-types.mlir
mlir/test/Dialect/Quant/quantile-types.mlir
Modified:
mlir/include/mlir/Dialect/Quant/IR/QuantTypes.h
mlir/lib/Dialect/Quant/IR/QuantOps.cpp
mlir/lib/Dialect/Quant/IR/QuantTypes.cpp
mlir/lib/Dialect/Quant/IR/TypeDetail.h
mlir/lib/Dialect/Quant/IR/TypeParser.cpp
mlir/test/Dialect/Quant/parse-uniform-invalid.mlir
mlir/test/Dialect/Quant/parse-uniform.mlir
Removed:
################################################################################
diff --git a/mlir/include/mlir/Dialect/Quant/IR/QuantTypes.h b/mlir/include/mlir/Dialect/Quant/IR/QuantTypes.h
index 34f47a15395c9..2001178d5939c 100644
--- a/mlir/include/mlir/Dialect/Quant/IR/QuantTypes.h
+++ b/mlir/include/mlir/Dialect/Quant/IR/QuantTypes.h
@@ -27,6 +27,7 @@ struct UniformQuantizedSubChannelTypeStorage;
struct UniformQuantizedTypeStorage;
struct UniformQuantizedPerAxisTypeStorage;
struct CalibratedQuantizedTypeStorage;
+struct QuantileTypeStorage;
} // namespace detail
@@ -255,7 +256,7 @@ class AnyQuantizedType
/// Per-layer, optional parameters omitted:
/// !quant<uniform[StorageType]{Scale}>
///
-/// StorageType: 'i'|'u' NumBits
+/// StorageType: 'i'|'u' NumBits, 'f4', 'F8E5M2', 'bf8', 'quantile'
/// ExpressedType: 'f16', 'f32', 'bf16', 'f64'
/// Scale: A legal double value
/// ZeroPoint: An integer value
@@ -313,7 +314,7 @@ class UniformQuantizedType
/// Per-axis, optional parameters omitted:
/// !quant<uniform[StorageType]{Scale}>
///
-/// StorageType: 'i'|'u' NumBits
+/// StorageType: 'i'|'u' NumBits, 'f4', 'hf8', 'bf8', 'quantile'
/// ExpressedType: 'f16', 'f32', 'bf16', 'f64'
/// QuantizedDim: An integer value
/// QuantParams: (Scale ':' ZeroPoint)+
@@ -398,7 +399,7 @@ class UniformQuantizedPerAxisType
/// ScaleZeroList ::= ScaleZero (',' ScaleZero)*
/// ScaleZero ::= Scale (':' ZeroPoint)?
///
-/// StorageType: 'i'|'u' NumBits
+/// StorageType: 'i'|'u' NumBits, 'f4', 'hf8', 'bf8', 'quantile'
/// ExpressedType: 'f16', 'f32', 'bf16', 'f64'
/// AxisSpec: An integer value
/// BlockSizeSpec: An integer value
@@ -547,6 +548,125 @@ class CalibratedQuantizedType
double getMax() const;
};
+/*Syntax:
+
+ ```
+ quantile-type ::= `!quant.quantile` `<` type `:` type `,` `{` float-list `}`
+ (`,` `<` int `,` int `>`)? `>`
+ ```
+
+ A quantile type represents a quantile-based floating point encoding, where
+ discrete storage values are totally defined by the floating-point values
+ entries in a quantile lookup table of F8/F16/F32/F64.
+
+ Optionally, explicit minimum and maximum storage values can be specified
+ after the LUT as `<min:max>`.
+
+ This type is used for weight compression schemes like NF4 (NormalizedFloat4)
+ and similar quantile-based formats.
+
+ Example:
+
+ MLIR:
+ !quant.quantile<ui4:f16, {-1.0,-0.696,0.0,0.079,1.0}>
+ !quant.quantile<ui4:f16, {-1.0,-0.696,0.0,0.079,1.0}, <-8,7>>
+
+ As an additional explanation for better understanding and readability of the
+ above example, the quantile type can be broken down as follows:
+ - `!quant.quantile`: This indicates that we are defining a quantile type.
+ - `<ui4:f16`: This specifies the storage type and the quantile type. In this
+ case, `ui4` indicates an unsigned 4-bit integer storage type, and `f16`
+ indicates that the quantile values are represented as 16-bit floating-point
+ numbers.
+ - `{-1.0,-0.696,0.0,0.079,1.0}`: This is the quantile lookup table (LUT)
+ that defines the discrete storage values. Each value in the LUT corresponds
+ to a specific quantized value that can be stored in the `ui4` storage type.
+ - `, <-8,7>`: This optional part specifies the explicit minimum and maximum
+ storage values. In this case, the minimum storage value is -8 and the maximum
+ storage value is 7.
+*/
+
+class QuantileType
+ : public Type::TypeBase<QuantileType, QuantizedType,
+ detail::QuantileTypeStorage,
+ mlir::QuantStorageTypeInterface::Trait> {
+public:
+ using ImplType = detail::QuantileTypeStorage;
+ using Base::Base;
+
+ // Get the underlying type used for to store raw values.
+ Type getStorageType() const;
+
+ // Get primitive expressed type of data in quantiles.
+ // Note that we may convert FP8 data to FP16 for storage,
+ // but we should treat its expressed type as FP8 rather than FP16.
+ Type getQuantileType() const;
+
+ /// Return the quantile table of this float type.
+ ArrayRef<double> getQuantiles() const;
+
+ /// Return the explicit storage minimum, if set.
+ std::optional<int64_t> getStorageMin() const;
+
+ /// Return the explicit storage maximum, if set.
+ std::optional<int64_t> getStorageMax() const;
+
+ // Get a quantile float type with specified quantile table.
+ static QuantileType get(mlir::MLIRContext *ctx, Type storageType,
+ Type quantileType, ArrayRef<double> quantiles = {},
+ std::optional<int64_t> storageMin = std::nullopt,
+ std::optional<int64_t> storageMax = std::nullopt);
+
+ static QuantileType
+ getChecked(function_ref<InFlightDiagnostic()> emitError,
+ mlir::MLIRContext *ctx, Type storageType, Type quantileType,
+ ArrayRef<double> quantiles,
+ std::optional<int64_t> storageMin = std::nullopt,
+ std::optional<int64_t> storageMax = std::nullopt);
+
+ static LogicalResult verifyInvariants(
+ function_ref<InFlightDiagnostic()> emitError, Type storageType,
+ Type quantileType, ArrayRef<double> quantiles,
+ std::optional<int64_t> storageMin, std::optional<int64_t> storageMax);
+
+ /// Methods for support type inquiry through isa, cast, and dyn_cast.
+ static bool classof(mlir::Type type);
+
+ // Printer
+ void print(mlir::AsmPrinter &printer) const;
+
+ static constexpr llvm::StringLiteral getMnemonic() { return {"quantile"}; }
+
+ static constexpr llvm::StringLiteral name = "quantile";
+
+ // Returns true if the type defaults to signed (e.g., si8, i8 or float types),
+ // false otherwise
+ bool shouldDefaultToSigned() const;
+
+ // Get the bit width of the storage type.
+ unsigned getStorageWidth() const;
+
+ // Get the default minimum and maximum values for the storage type.
+ int64_t getDefaultMinimum([[maybe_unused]] bool isSigned) const;
+ int64_t getDefaultMaximum([[maybe_unused]] bool isSigned) const;
+
+ // Get the string representation of the storage type
+ std::string getStorageTypeName([[maybe_unused]] bool isSigned) const;
+
+ // Get whether the type is a packed quantile float type
+ bool isPacked() const;
+
+ // Get the logical bit width of the quantile float type, which is the bit
+ // width of the represented floating point value.
+ unsigned getLogicalBitWidth() const;
+
+ // Get the number of quantized values stored in one byte for this quantile
+ // float type.
+ unsigned getElementsPerByte() const;
+
+ // Get the preferred alignment in bytes for this quantile float type, if any.
+ std::optional<unsigned> getPreferredAlignmentBytes() const;
+};
} // namespace quant
} // namespace mlir
diff --git a/mlir/lib/Dialect/Quant/IR/QuantOps.cpp b/mlir/lib/Dialect/Quant/IR/QuantOps.cpp
index 060707437334e..36e660d672381 100644
--- a/mlir/lib/Dialect/Quant/IR/QuantOps.cpp
+++ b/mlir/lib/Dialect/Quant/IR/QuantOps.cpp
@@ -200,7 +200,8 @@ struct QuantInlinerInterface : public DialectInlinerInterface {
void QuantDialect::initialize() {
addTypes<AnyQuantizedType, CalibratedQuantizedType, UniformQuantizedType,
- UniformQuantizedPerAxisType, UniformQuantizedSubChannelType>();
+ UniformQuantizedPerAxisType, UniformQuantizedSubChannelType,
+ QuantileType>();
addOperations<
#define GET_OP_LIST
#include "mlir/Dialect/Quant/IR/QuantOps.cpp.inc"
diff --git a/mlir/lib/Dialect/Quant/IR/QuantTypes.cpp b/mlir/lib/Dialect/Quant/IR/QuantTypes.cpp
index c5a36f7106ad3..c150c151e1e11 100644
--- a/mlir/lib/Dialect/Quant/IR/QuantTypes.cpp
+++ b/mlir/lib/Dialect/Quant/IR/QuantTypes.cpp
@@ -9,6 +9,8 @@
#include "mlir/Dialect/Quant/IR/QuantTypes.h"
#include "TypeDetail.h"
#include "mlir/Dialect/Quant/IR/Quant.h"
+#include "mlir/IR/DialectImplementation.h"
+#include "mlir/IR/OpImplementation.h"
#include "mlir/IR/QuantStorageTypeInterface.h"
#include "mlir/IR/BuiltinTypes.h"
@@ -552,3 +554,139 @@ LogicalResult CalibratedQuantizedType::verifyInvariants(
double CalibratedQuantizedType::getMin() const { return getImpl()->min; }
double CalibratedQuantizedType::getMax() const { return getImpl()->max; }
+
+QuantileType QuantileType::get(mlir::MLIRContext *ctx, mlir::Type storageType,
+ mlir::Type quantileType,
+ ArrayRef<double> quantiles,
+ std::optional<int64_t> storageMin,
+ std::optional<int64_t> storageMax) {
+ return Base::get(ctx, storageType, quantileType, quantiles, storageMin,
+ storageMax);
+}
+
+QuantileType QuantileType::getChecked(
+ function_ref<InFlightDiagnostic()> emitError, mlir::MLIRContext *ctx,
+ mlir::Type storageType, mlir::Type quantileType, ArrayRef<double> quantiles,
+ std::optional<int64_t> storageMin, std::optional<int64_t> storageMax) {
+ return Base::getChecked(emitError, ctx, storageType, quantileType, quantiles,
+ storageMin, storageMax);
+}
+
+LogicalResult QuantileType::verifyInvariants(
+ function_ref<InFlightDiagnostic()> emitError, Type storageType,
+ Type quantileType, ArrayRef<double> quantiles,
+ std::optional<int64_t> storageMin, std::optional<int64_t> storageMax) {
+ if (!storageType.isIntOrFloat())
+ return emitError() << "storage type must be an integer or float type";
+ if (!llvm::isa<mlir::FloatType>(quantileType))
+ return emitError() << "quantile type must be a float type";
+ if (quantiles.empty())
+ return emitError() << "quantile values must not be empty";
+ if (storageMin.has_value() != storageMax.has_value())
+ return emitError()
+ << "storage min and max must both be specified or both omitted";
+ if (storageMin && storageMax && *storageMin >= *storageMax)
+ return emitError() << "storage min must be less than storage max";
+
+ unsigned width = storageType.getIntOrFloatBitWidth();
+ bool isSigned = !llvm::isa<mlir::IntegerType>(storageType) ||
+ llvm::cast<mlir::IntegerType>(storageType).isSigned();
+ auto effectiveMin =
+ storageMin.value_or(isSigned ? -(1LL << (width - 1)) : 0LL);
+ auto effectiveMax = storageMax.value_or(isSigned ? (1LL << (width - 1)) - 1
+ : (1LL << width) - 1);
+ auto expectedSize = effectiveMax - effectiveMin + 1;
+ if (static_cast<decltype(expectedSize)>(quantiles.size()) != expectedSize)
+ return emitError() << "quantile LUT size (" << quantiles.size()
+ << ") must equal the number of representable storage "
+ "values ("
+ << expectedSize << ")";
+
+ for (double v : quantiles)
+ if (std::isnan(v) || std::isinf(v))
+ return emitError()
+ << "quantile values must be finite (no NaN or infinity)";
+
+ return success();
+}
+
+bool QuantileType::classof(mlir::Type type) {
+ return type.getTypeID() == mlir::TypeID::get<QuantileType>();
+}
+
+mlir::Type QuantileType::getStorageType() const {
+ return static_cast<ImplType *>(impl)->getStorageType();
+}
+
+mlir::Type QuantileType::getQuantileType() const {
+ return static_cast<ImplType *>(impl)->getQuantileType();
+}
+
+ArrayRef<double> QuantileType::getQuantiles() const {
+ return static_cast<ImplType *>(impl)->getQuantiles();
+}
+
+std::optional<int64_t> QuantileType::getStorageMin() const {
+ return static_cast<ImplType *>(impl)->getStorageMin();
+}
+
+std::optional<int64_t> QuantileType::getStorageMax() const {
+ return static_cast<ImplType *>(impl)->getStorageMax();
+}
+
+bool QuantileType::shouldDefaultToSigned() const {
+ if (auto intType = mlir::dyn_cast<mlir::IntegerType>(getStorageType()))
+ return intType.isSigned();
+ // Float types default to signed.
+ return true;
+}
+
+unsigned QuantileType::getStorageWidth() const {
+ return getStorageType().getIntOrFloatBitWidth();
+}
+
+int64_t QuantileType::getDefaultMaximum(bool isSigned) const {
+ if (auto explicitMax = getStorageMax())
+ return *explicitMax;
+ if (isSigned)
+ return (1LL << (getStorageWidth() - 1)) - 1;
+ return (1LL << getStorageWidth()) - 1;
+}
+
+int64_t QuantileType::getDefaultMinimum(bool isSigned) const {
+ if (auto explicitMin = getStorageMin())
+ return *explicitMin;
+ if (isSigned)
+ return -(1LL << (getStorageWidth() - 1));
+ return 0;
+}
+
+std::string QuantileType::getStorageTypeName(bool isSigned) const {
+ std::string result = "!quant.quantile<";
+ llvm::raw_string_ostream os(result);
+ os << getStorageType() << ":" << getQuantileType() << ", {";
+ ArrayRef<double> quantiles = this->getQuantiles();
+ llvm::interleave(
+ llvm::seq<size_t>(0, quantiles.size()), os,
+ [&](size_t index) { os << quantiles[index]; }, ",");
+ os << "}";
+ if (auto minVal = getStorageMin())
+ if (auto maxVal = getStorageMax())
+ os << ", <" << *minVal << ":" << *maxVal << ">";
+ os << ">";
+ os.flush();
+ return result;
+}
+
+bool QuantileType::isPacked() const { return getStorageWidth() <= 4; }
+
+unsigned QuantileType::getLogicalBitWidth() const { return getStorageWidth(); }
+
+unsigned QuantileType::getElementsPerByte() const {
+ unsigned width = getStorageWidth();
+ return width > 0 ? 8 / width : 0;
+}
+
+std::optional<unsigned> QuantileType::getPreferredAlignmentBytes() const {
+ return std::nullopt;
+}
diff --git a/mlir/lib/Dialect/Quant/IR/TypeDetail.h b/mlir/lib/Dialect/Quant/IR/TypeDetail.h
index a43bce354c324..8834ee7901c18 100644
--- a/mlir/lib/Dialect/Quant/IR/TypeDetail.h
+++ b/mlir/lib/Dialect/Quant/IR/TypeDetail.h
@@ -422,6 +422,86 @@ struct CalibratedQuantizedTypeStorage : public QuantizedTypeStorage {
double max;
};
+struct QuantileTypeStorage : public mlir::TypeStorage {
+ mlir::Type storageType;
+ mlir::Type quantileType;
+ const double *quantilesElements;
+ size_t quantilesParamsSize;
+ std::optional<int64_t> storageMin;
+ std::optional<int64_t> storageMax;
+
+ struct KeyTy {
+ KeyTy(mlir::Type storageType, mlir::Type quantileType,
+ ArrayRef<double> quantiles,
+ std::optional<int64_t> storageMin = std::nullopt,
+ std::optional<int64_t> storageMax = std::nullopt)
+ : storageType(storageType), quantileType(quantileType),
+ quantiles(quantiles), storageMin(storageMin), storageMax(storageMax) {
+ }
+
+ mlir::Type storageType;
+ mlir::Type quantileType;
+ ArrayRef<double> quantiles;
+ std::optional<int64_t> storageMin;
+ std::optional<int64_t> storageMax;
+
+ mlir::Type getQuantileType() const { return quantileType; }
+ ArrayRef<double> getQuantiles() const { return quantiles; }
+
+ bool operator==(const KeyTy &other) const {
+ return storageType == other.storageType &&
+ quantileType == other.quantileType &&
+ quantiles == other.quantiles && storageMin == other.storageMin &&
+ storageMax == other.storageMax;
+ }
+
+ static llvm::hash_code hashOptInt(std::optional<int64_t> opt) {
+ return opt ? llvm::hash_combine(true, *opt)
+ : llvm::hash_combine(false, int64_t{0});
+ }
+
+ unsigned getHashValue() const {
+ const int64_t *quantilesCast =
+ llvm::bit_cast<const int64_t *>(quantiles.data());
+ ArrayRef<int64_t> quantilesBits(quantilesCast, quantiles.size());
+ return static_cast<unsigned>(llvm::hash_combine(
+ llvm::hash_combine_range(quantilesBits.begin(), quantilesBits.end()),
+ storageType, quantileType, hashOptInt(storageMin),
+ hashOptInt(storageMax)));
+ }
+ };
+
+ bool operator==(const KeyTy &key) const {
+ return storageType == key.storageType && quantileType == key.quantileType &&
+ getQuantiles() == key.quantiles && storageMin == key.storageMin &&
+ storageMax == key.storageMax;
+ }
+
+ QuantileTypeStorage(const KeyTy &key, ArrayRef<double> quantiles)
+ : storageType(key.storageType), quantileType(key.quantileType),
+ quantilesElements(quantiles.data()),
+ quantilesParamsSize(quantiles.size()), storageMin(key.storageMin),
+ storageMax(key.storageMax) {}
+
+ static QuantileTypeStorage *construct(mlir::TypeStorageAllocator &allocator,
+ KeyTy key) {
+ ArrayRef<double> quantiles = allocator.copyInto(key.quantiles);
+ return new (allocator.allocate<QuantileTypeStorage>())
+ QuantileTypeStorage(key, quantiles);
+ }
+
+ static unsigned hashKey(const KeyTy &key) { return key.getHashValue(); }
+
+ ArrayRef<double> getQuantiles() const {
+ return ArrayRef<double>(quantilesElements, quantilesParamsSize);
+ }
+
+ mlir::Type getStorageType() const { return storageType; }
+ mlir::Type getQuantileType() const { return quantileType; }
+ std::optional<int64_t> getStorageMin() const { return storageMin; }
+ std::optional<int64_t> getStorageMax() const { return storageMax; }
+};
+
} // namespace detail
} // namespace quant
} // namespace mlir
diff --git a/mlir/lib/Dialect/Quant/IR/TypeParser.cpp b/mlir/lib/Dialect/Quant/IR/TypeParser.cpp
index 1a42b90ac31e2..2a845bed0a535 100644
--- a/mlir/lib/Dialect/Quant/IR/TypeParser.cpp
+++ b/mlir/lib/Dialect/Quant/IR/TypeParser.cpp
@@ -320,7 +320,8 @@ parseQuantParamListUntilRBrace(DialectAsmParser &parser, Type expressedType,
/// block-size-info `,` scale-zero-tensor `>`
/// storage-spec ::= storage-type (`<` storage-range `>`)?
/// storage-range ::= integer-literal `:` integer-literal
-/// storage-type ::= (`i` | `u`) integer-literal
+/// storage-type ::= (`i` | `u`) integer-literal | `f8E5M2` | `f8E4M3FN`
+// | `f4E2M1FN` | 'quantile'
/// expressed-type-spec ::= `:` `f` integer-literal
/// axis-spec ::= `:` integer-literal
/// scale-zero ::= scale (`:` zero-point)?
@@ -479,6 +480,60 @@ static Type parseCalibratedType(DialectAsmParser &parser) {
return parser.getChecked<CalibratedQuantizedType>(expressedType, min, max);
}
+static Type parseQuantileType(DialectAsmParser &parser) {
+ Type storageType;
+ Type quantileType;
+ SmallVector<double, 1> quantiles;
+
+ if (parser.parseLess())
+ return nullptr;
+ if (parser.parseType(storageType))
+ return nullptr;
+ if (parser.parseColon())
+ return nullptr;
+ if (parser.parseType(quantileType))
+ return nullptr;
+ if (parser.parseComma())
+ return nullptr;
+ if (parser.parseLBrace())
+ return nullptr;
+
+ // Allow empty braces `{}` — verify() will catch the empty quantile error.
+ if (failed(parser.parseOptionalRBrace())) {
+ do {
+ quantiles.emplace_back();
+ if (parser.parseFloat(quantiles.back()))
+ return nullptr;
+ } while (succeeded(parser.parseOptionalComma()));
+
+ if (parser.parseRBrace())
+ return nullptr;
+ }
+
+ // Optionally parse explicit storage range: `, min:max` (inside the outer
+ // `<>`).
+ std::optional<int64_t> storageMin, storageMax;
+ if (succeeded(parser.parseOptionalComma())) {
+ if (parser.parseLess())
+ return nullptr;
+ int64_t minVal, maxVal;
+ if (parser.parseInteger(minVal) || parser.parseColon() ||
+ parser.parseInteger(maxVal))
+ return nullptr;
+ storageMin = minVal;
+ storageMax = maxVal;
+ if (parser.parseGreater())
+ return nullptr;
+ }
+
+ if (parser.parseGreater())
+ return nullptr;
+
+ mlir::MLIRContext *ctx = parser.getContext();
+ return parser.getChecked<QuantileType>(ctx, storageType, quantileType,
+ quantiles, storageMin, storageMax);
+}
+
/// Parse a type registered to this dialect.
Type QuantDialect::parseType(DialectAsmParser &parser) const {
// All types start with an identifier that we switch on.
@@ -492,6 +547,8 @@ Type QuantDialect::parseType(DialectAsmParser &parser) const {
return parseAnyType(parser);
if (typeNameSpelling == "calibrated")
return parseCalibratedType(parser);
+ if (typeNameSpelling == "quantile")
+ return parseQuantileType(parser);
parser.emitError(parser.getNameLoc(),
"unknown quantized type " + typeNameSpelling);
@@ -652,6 +709,23 @@ static void printCalibratedQuantizedType(CalibratedQuantizedType type,
out << ">";
}
+static void printQuantileType(QuantileType type, DialectAsmPrinter &out) {
+ out << "quantile<";
+ out << type.getStorageType();
+ out << ":";
+ out << type.getQuantileType();
+ out << ", {";
+ ArrayRef<double> quantiles = type.getQuantiles();
+ llvm::interleave(
+ llvm::seq<size_t>(0, quantiles.size()), out,
+ [&](size_t index) { out << quantiles[index]; }, ",");
+ out << "}";
+ if (auto minVal = type.getStorageMin())
+ if (auto maxVal = type.getStorageMax())
+ out << ", <" << *minVal << ":" << *maxVal << ">";
+ out << ">";
+}
+
/// Print a type registered to this dialect.
void QuantDialect::printType(Type type, DialectAsmPrinter &os) const {
if (auto anyType = llvm::dyn_cast<AnyQuantizedType>(type))
@@ -665,6 +739,8 @@ void QuantDialect::printType(Type type, DialectAsmPrinter &os) const {
printUniformQuantizedSubChannelType(perAxisType, os);
else if (auto calibratedType = llvm::dyn_cast<CalibratedQuantizedType>(type))
printCalibratedQuantizedType(calibratedType, os);
+ else if (auto quantileType = llvm::dyn_cast<QuantileType>(type))
+ printQuantileType(quantileType, os);
else
llvm_unreachable("Unhandled quantized type");
}
diff --git a/mlir/test/Dialect/Quant/invalid-quantile-types.mlir b/mlir/test/Dialect/Quant/invalid-quantile-types.mlir
new file mode 100644
index 0000000000000..faf16d01a9cd5
--- /dev/null
+++ b/mlir/test/Dialect/Quant/invalid-quantile-types.mlir
@@ -0,0 +1,42 @@
+// RUN: mlir-opt -allow-unregistered-dialect %s -split-input-file -verify-diagnostics
+
+//===----------------------------------------------------------------------===//
+// Verify errors (caught by verify(), reached through getChecked())
+//===----------------------------------------------------------------------===//
+
+// Storage type must be an integer or float.
+// expected-error @+1 {{storage type must be an integer or float type}}
+func.func private @invalid_storage_type() -> !quant.quantile<tensor<1xf32>:f32, {1.0}>
+
+// -----
+
+// Quantile (expressed) type must be a float.
+// expected-error @+1 {{quantile type must be a float type}}
+func.func private @invalid_quantile_type() -> !quant.quantile<ui4:i8, {1.0, 0.0, -1.0}>
+
+// -----
+
+// Quantile LUT must not be empty.
+// expected-error @+1 {{quantile values must not be empty}}
+func.func private @empty_quantiles() -> !quant.quantile<ui4:f16, {}>
+
+// -----
+
+// LUT size must match the number of representable storage values.
+// ui4 has 16 representable values [0,15], but only 3 are provided.
+// expected-error @+1 {{quantile LUT size (3) must equal the number of representable storage values (16)}}
+func.func private @wrong_lut_size() -> !quant.quantile<ui4:f16, {-1.0,0.0,1.0}>
+
+// -----
+
+// Explicit storage range: min must be strictly less than max.
+// si4 default range is [-8,7]; explicit 5:3 has min > max.
+// expected-error @+1 {{storage min must be less than storage max}}
+func.func private @invalid_range_order() -> !quant.quantile<si4:f32, {-2.0,-1.875,-1.75,-1.625,-1.5,-1.375,-1.25,-1.125,-1.0,-0.875,-0.75,-0.625,-0.5,-0.375,-0.25,-0.125}, <5:3>>
+
+// -----
+
+// LUT size must match the total representable values of the storage type.
+// f4E2M1FN has 16 representable values regardless of explicit range, but only 3 are provided.
+// expected-error @+1 {{quantile LUT size (3) must equal the number of representable storage values (13)}}
+func.func private @wrong_lut_size_with_range() -> !quant.quantile<f4E2M1FN:f16, {-1.0,0.0,1.0}, <-6:6>>
diff --git a/mlir/test/Dialect/Quant/parse-uniform-invalid.mlir b/mlir/test/Dialect/Quant/parse-uniform-invalid.mlir
index 6dbc86263bd71..5ff54a8804844 100644
--- a/mlir/test/Dialect/Quant/parse-uniform-invalid.mlir
+++ b/mlir/test/Dialect/Quant/parse-uniform-invalid.mlir
@@ -264,3 +264,21 @@
// Illegal storage min/max: min < defaultMin
// expected-error at +1 {{illegal storage type minimum: -10}}
!qalias = !quant.uniform<f4E2M1FN<-10:6>:f32, 0.99872:127>
+
+// -----
+
+// Invalid LUT size: 16 values but explicit range 6:10 has only 5 representable values.
+// expected-error at +1 {{quantile LUT size (16) must equal the number of representable storage values (5)}}
+!qalias = !quant.uniform<!quant.quantile<f4E2M1FN:f16, {-1.0000,-0.8667,-0.7333,-0.6000,-0.4667,-0.3333,-0.2000,-0.0667,0.0667,0.2000,0.3333,0.4667,0.6000,0.7333,0.8667,1.0000}, <6:10>>:f32, 0.99872:127>
+
+// -----
+
+// Quantile storage range: min must be strictly less than max.
+// expected-error at +1 {{storage min must be less than storage max}}
+!qalias = !quant.uniform<!quant.quantile<f4E2M1FN:f16,{-1.0000,-0.8667,-0.7333,-0.6000,-0.4667,-0.3333,-0.2000,-0.0667,0.0667,0.2000,0.3333,0.4667,0.6000,0.7333,0.8667,1.0000}, <5:3>>:f32, 0.99872:127>
+
+// -----
+
+// Quantile LUT size (3) does not match the 16 representable values of f4E2M1FN's default range.
+// expected-error at +1 {{quantile LUT size (3) must equal the number of representable storage values (16)}}
+!qalias = !quant.uniform<!quant.quantile<f4E2M1FN:f16, {-1.0,0.0,1.0}>:f32, 0.99872:127>
diff --git a/mlir/test/Dialect/Quant/parse-uniform.mlir b/mlir/test/Dialect/Quant/parse-uniform.mlir
index a8b9e5707b474..56219d2a3f437 100644
--- a/mlir/test/Dialect/Quant/parse-uniform.mlir
+++ b/mlir/test/Dialect/Quant/parse-uniform.mlir
@@ -245,3 +245,22 @@ func.func @parse() -> !qalias {
%0 = "foo"() : () -> !qalias
return %0 : !qalias
}
+
+// -----
+// Storage type: QuantileType with narrowed explicit range `-6:6` (13 representable values).
+// CHECK: !quant.uniform<!quant.quantile<f4E2M1FN:f16, {-1.000000e+00,-8.750000e-01,-7.500000e-01,-6.250000e-01,-5.000000e-01,-2.500000e-01,0.000000e+00,2.500000e-01,5.000000e-01,6.250000e-01,7.500000e-01,8.750000e-01,1.000000e+00}, <-6:6>>:f32, 9.987200e-01:127>
+!qalias = !quant.uniform<!quant.quantile<f4E2M1FN:f16, {-1.0,-0.875,-0.75,-0.625,-0.5,-0.25,0.0,0.25,0.5,0.625,0.75,0.875,1.0}, <-6:6>>:f32, 0.99872:127>
+func.func @parse() -> !qalias {
+ %0 = "foo"() : () -> !qalias
+ return %0 : !qalias
+}
+
+// -----
+// Storage type: QuantileType
+// CHECK: !quant.uniform<!quant.quantile<f4E2M1FN:f16, {
+!qalias = !quant.uniform<!quant.quantile<f4E2M1FN:f16, {-1.0000,-0.8667,-0.7333,-0.6000,-0.4667,-0.3333,-0.2000,-0.0667,0.0667,0.2000,0.3333,0.4667,0.6000,0.7333,0.8667,1.0000}>:f32, 2.0e+2 >
+func.func @parse() -> !qalias {
+ %0 = "foo"() : () -> !qalias
+ return %0 : !qalias
+}
+
diff --git a/mlir/test/Dialect/Quant/quantile-types.mlir b/mlir/test/Dialect/Quant/quantile-types.mlir
new file mode 100644
index 0000000000000..c8840a7b3d3f2
--- /dev/null
+++ b/mlir/test/Dialect/Quant/quantile-types.mlir
@@ -0,0 +1,76 @@
+// RUN: mlir-opt -allow-unregistered-dialect %s -split-input-file | FileCheck %s
+
+// -----
+// Quantile type: ui4 storage-type with f16 expressed-type, 16 entries (default range 0..15).
+// CHECK-LABEL: func private @quantile_ui4_f16
+// CHECK-SAME: !quant.quantile<ui4:f16, {
+func.func private @quantile_ui4_f16(!quant.quantile<ui4:f16, {-1.0,-0.8667,-0.7333,-0.6,-0.4667,-0.3333,-0.2,-0.0667,0.0667,0.2,0.3333,0.4667,0.6,0.7333,0.8667,1.0}>) -> ()
+
+// -----
+// Quantile type: si8 storage-type with f32 expressed-type, explicit range -2:2 (5 entries).
+// CHECK: func private @quantile_si8_f32(!quant.quantile<si8:f32, {-1.000000e+00,-5.000000e-01,0.000000e+00,5.000000e-01,1.000000e+00}, <-2:2>>)
+func.func private @quantile_si8_f32(!quant.quantile<si8:f32, {-1.0,-0.5,0.0,0.5,1.0}, <-2:2>>) -> ()
+
+// -----
+// Quantile type: i8 (signless) storage-type with f32 expressed-type, explicit range -1:1 (3 entries).
+// CHECK: func private @quantile_i8_f32(!quant.quantile<i8:f32, {-1.000000e+00,0.000000e+00,1.000000e+00}, <-1:1>>)
+func.func private @quantile_i8_f32(!quant.quantile<i8:f32, {-1.0,0.0,1.0}, <-1:1>>) -> ()
+
+// -----
+// Quantile type: f8E4M3FN float storage-type with f32 expressed-type, explicit range -1:1 (3 entries).
+// CHECK: func private @quantile_f8_f32(!quant.quantile<f8E4M3FN:f32, {-1.000000e+00,0.000000e+00,1.000000e+00}, <-1:1>>)
+func.func private @quantile_f8_f32(!quant.quantile<f8E4M3FN:f32, {-1.0,0.0,1.0}, <-1:1>>) -> ()
+
+// -----
+// Quantile type: ui4 storage-type with bf16 expressed-type, 16 entries.
+// CHECK-LABEL: func private @quantile_ui4_bf16
+// CHECK-SAME: !quant.quantile<ui4:bf16, {
+func.func private @quantile_ui4_bf16(!quant.quantile<ui4:bf16, {-1.0,-0.8667,-0.7333,-0.6,-0.4667,-0.3333,-0.2,-0.0667,0.0667,0.2,0.3333,0.4667,0.6,0.7333,0.8667,1.0}>) -> ()
+
+// -----
+// Quantile type used as a return type.
+// CHECK-LABEL: func private @quantile_as_return
+// CHECK-SAME: !quant.quantile<ui4:f16, {
+func.func private @quantile_as_return() -> !quant.quantile<ui4:f16, {-1.0,-0.8667,-0.7333,-0.6,-0.4667,-0.3333,-0.2,-0.0667,0.0667,0.2,0.3333,0.4667,0.6,0.7333,0.8667,1.0}>
+
+// -----
+// NF4-style 16-entry quantile table.
+// CHECK-LABEL: @nf4_16_values
+// CHECK-SAME: !quant.quantile<ui4:f16, {
+func.func private @nf4_16_values(!quant.quantile<ui4:f16, {
+ -1.0,-0.6961928009986877,-0.5250730514526367,-0.39491748809814453,
+ -0.28444138169288635,-0.18477343022823334,-0.09105003625154495,0.0,
+ 0.07958029955625534,0.16093020141124725,0.24611230194568634,
+ 0.33791524171829224,0.44070982933044434,0.5626170039176941,
+ 0.7229568362236023,1.0}>) -> ()
+
+// -----
+// Explicit storage min/max range (unsigned storage, narrowed range 0..7, 8 entries).
+// CHECK: func private @quantile_with_range(!quant.quantile<ui4:f16, {-1.000000e+00,-7.500000e-01,-5.000000e-01,-2.500000e-01,0.000000e+00,2.500000e-01,5.000000e-01,1.000000e+00}, <0:7>>)
+func.func private @quantile_with_range(!quant.quantile<ui4:f16, {-1.0,-0.75,-0.5,-0.25,0.0,0.25,0.5,1.0}, <0:7>>) -> ()
+
+// -----
+// Explicit range is preserved through round-trip.
+// CHECK: func private @quantile_signed_range(!quant.quantile<si8:f32, {-1.000000e+00,0.000000e+00,1.000000e+00}, <-1:1>>)
+func.func private @quantile_signed_range(!quant.quantile<si8:f32, {-1.0,0.0,1.0}, <-1:1>>) -> ()
+
+// -----
+// Signed 4-bit storage-type uses full 16-entry LUT (range -8..7).
+// CHECK-LABEL: func private @quantile_negatives
+// CHECK-SAME: !quant.quantile<si4:f32, {-2.000000e+00,-1.875000e+00,-1.750000e+00,-1.625000e+00,-1.500000e+00,-1.375000e+00,-1.250000e+00,-1.125000e+00,-1.000000e+00,-8.750000e-01,-7.500000e-01,-6.250000e-01,-5.000000e-01,-3.750000e-01,-2.500000e-01,-1.250000e-01}>
+func.func private @quantile_negatives(!quant.quantile<si4:f32, {-2.0,-1.875,-1.75,-1.625,-1.5,-1.375,-1.25,-1.125,-1.0,-0.875,-0.75,-0.625,-0.5,-0.375,-0.25,-0.125}>) -> ()
+
+// -----
+// 1-bit unsigned storage: minimal 2-entry LUT.
+// CHECK: func private @quantile_ui1_f16(!quant.quantile<ui1:f16, {-1.000000e+00,1.000000e+00}>)
+func.func private @quantile_ui1_f16(!quant.quantile<ui1:f16, {-1.0,1.0}>) -> ()
+
+// -----
+// LUT values in descending order (ui4, explicit range 0:7, 8 entries).
+// CHECK: func private @quantile_descending(!quant.quantile<ui4:f16, {1.000000e+00,7.500000e-01,5.000000e-01,2.500000e-01,0.000000e+00,-2.500000e-01,-5.000000e-01,-1.000000e+00}, <0:7>>)
+func.func private @quantile_descending(!quant.quantile<ui4:f16, {1.0,0.75,0.5,0.25,0.0,-0.25,-0.5,-1.0}, <0:7>>) -> ()
+
+// -----
+// LUT values in arbitrary order (ui4, explicit range 0:7, 8 entries).
+// CHECK: func private @quantile_random_order(!quant.quantile<ui4:f16, {0.000000e+00,-5.000000e-01,1.000000e+00,-2.500000e-01,7.500000e-01,-1.000000e+00,5.000000e-01,2.500000e-01}, <0:7>>)
+func.func private @quantile_random_order(!quant.quantile<ui4:f16, {0.0,-0.5,1.0,-0.25,0.75,-1.0,0.5,0.25}, <0:7>>) -> ()
More information about the Mlir-commits
mailing list