[Mlir-commits] [mlir] Sub-channel quantized type implementation (PR #120172)

[Sandeep Dasgupta]

================
@@ -159,46 +159,183 @@ static Type parseAnyType(DialectAsmParser &parser) {
       typeFlags, storageType, expressedType, storageTypeMin, storageTypeMax);
 }
 
-static ParseResult parseQuantParams(DialectAsmParser &parser, double &scale,
+/// Checks if the given scale value is within the valid range of the expressed
+/// type. The `expressedType` argument is the floating-point type used for
+/// expressing the quantized values, and `scale` is the double value to check.
+LogicalResult
+isScaleInExpressedTypeRange(function_ref<InFlightDiagnostic()> emitError,
----------------
sdasgup3 wrote:

The reason this sub-channel type check has to be moved from the verifier to the parser is because scales are expressed as DenseElementsAttr in its ctor. When DenseElementsAttr is created using double values that cannot be expressed in expressed_type, they are silently rounded to the nearest value in that type. This rounding prevents verification from occurring. This was not an issue for other quant types (like per-tensor, per-axis) because they take the double in the constructor, allowing for later verification. Had we choose to maintain the current behavior, sub-channel verification must occur in the parser and other verification in the verifier, resulting in duplicated code.

https://github.com/llvm/llvm-project/pull/120172