[llvm] [LoopIdiomRecognize] Enable clmul optimization for CRC loops (PR #203405)
Ramkumar Ramachandra via llvm-commits
llvm-commits at lists.llvm.org
Fri Jun 12 23:21:38 PDT 2026
================
@@ -376,6 +376,54 @@ CRCTable HashRecognize::genSarwateTable(const APInt &GenPoly,
return Table;
}
+// Divide one GF(2) polynomial by another.
+static APInt calculateGF2Quotient(APInt Dividend, const APInt &Divisor) {
+ unsigned DivisorDeg = Divisor.getActiveBits() - 1;
+ APInt Quotient = APInt::getZero(Dividend.getBitWidth());
+ unsigned DividendDeg;
+ while (!Dividend.isZero() &&
+ (DividendDeg = Dividend.getActiveBits() - 1) >= DivisorDeg) {
+ unsigned Shift = DividendDeg - DivisorDeg;
+ Quotient.setBit(Shift);
+ Dividend ^= Divisor.shl(Shift);
+ }
+ return Quotient;
+}
+
+// Generate constants (mu/reciprocal, P/generator) for a Barrett-style
+// reduction. This reduction allows the Sarwate table entry to be computed on
+// the fly, rather than requiring a load from memory (on supporting hardware).
+CRCBarrettConstants HashRecognize::genBarrettConstants(const APInt &GenPoly,
+ bool ByteOrderSwapped) {
+ unsigned BW = GenPoly.getBitWidth();
+ unsigned ClmulBW = BW * 2;
+ unsigned DivBW = ClmulBW + 1;
+ APInt Dividend = APInt::getSignedMinValue(DivBW);
+ CRCBarrettConstants C;
+
+ if (ByteOrderSwapped) {
+ APInt G = GenPoly.zext(DivBW);
+ G.setBit(BW);
+ APInt Mu = calculateGF2Quotient(Dividend, G);
+ C.Reciprocal = Mu.trunc(ClmulBW);
+ C.Generator = G.trunc(ClmulBW);
+ return C;
+ }
+
+ APInt G = GenPoly.reverseBits().zext(DivBW);
+ G.setBit(BW);
+ APInt Mu = calculateGF2Quotient(Dividend, G);
+ // Only the low byte of the reciprocal is useful in LE because it will get
+ // multiplied by another byte and then immediately masked by 0xFF.
+ C.Reciprocal = Mu.trunc(BW + 1).reverseBits().zext(ClmulBW).getLoBits(8);
+ // shl(1) here enables an optimization where two shifts then XOR can be
+ // replaced with an XOR then one shift. Normally one of the shifts would be by
+ // 7 and the other by 8, but by shifting here the shift by 7 becomes another
+ // shift by 8.
----------------
artagnon wrote:
The correct reason for shl(1) is because AArch64 CRC operates on bit-reflected data. The ASL code from the [manual](https://developer.arm.com/documentation/111183/2025-09_ASL1/Base-Instructions/CRC32--CRC32-?lang=en) specifies this:
```
bits(32) acc = X[n]; // accumulator
bits(size) val = X[m]; // input value
bits(32) poly = 0x04C11DB7;
bits(32+size) tempacc = BitReverse(acc):Zeros(size);
bits(size+32) tempval = BitReverse(val):Zeros(32);
// Poly32Mod2 on a bitstring does a polynomial Modulus over {0,1} operation
X[d] = BitReverse(Poly32Mod2(tempacc EOR tempval, poly));
```
However, llvm.clmul is not bit-reflected, and this shl(1) would be incorrect.
https://github.com/llvm/llvm-project/pull/203405
More information about the llvm-commits
mailing list