[llvm] [LoopIdiomRecognize] Enable clmul optimization for CRC loops (PR #203405)

Ramkumar Ramachandra via llvm-commits llvm-commits at lists.llvm.org
Fri Jun 12 23:21:38 PDT 2026


================
@@ -376,6 +376,54 @@ CRCTable HashRecognize::genSarwateTable(const APInt &GenPoly,
   return Table;
 }
 
+// Divide one GF(2) polynomial by another.
+static APInt calculateGF2Quotient(APInt Dividend, const APInt &Divisor) {
+  unsigned DivisorDeg = Divisor.getActiveBits() - 1;
+  APInt Quotient = APInt::getZero(Dividend.getBitWidth());
+  unsigned DividendDeg;
+  while (!Dividend.isZero() &&
+         (DividendDeg = Dividend.getActiveBits() - 1) >= DivisorDeg) {
+    unsigned Shift = DividendDeg - DivisorDeg;
+    Quotient.setBit(Shift);
+    Dividend ^= Divisor.shl(Shift);
+  }
+  return Quotient;
+}
+
+// Generate constants (mu/reciprocal, P/generator) for a Barrett-style
+// reduction. This reduction allows the Sarwate table entry to be computed on
+// the fly, rather than requiring a load from memory (on supporting hardware).
+CRCBarrettConstants HashRecognize::genBarrettConstants(const APInt &GenPoly,
+                                                       bool ByteOrderSwapped) {
+  unsigned BW = GenPoly.getBitWidth();
+  unsigned ClmulBW = BW * 2;
+  unsigned DivBW = ClmulBW + 1;
+  APInt Dividend = APInt::getSignedMinValue(DivBW);
+  CRCBarrettConstants C;
+
+  if (ByteOrderSwapped) {
+    APInt G = GenPoly.zext(DivBW);
+    G.setBit(BW);
+    APInt Mu = calculateGF2Quotient(Dividend, G);
+    C.Reciprocal = Mu.trunc(ClmulBW);
+    C.Generator = G.trunc(ClmulBW);
+    return C;
+  }
+
+  APInt G = GenPoly.reverseBits().zext(DivBW);
+  G.setBit(BW);
+  APInt Mu = calculateGF2Quotient(Dividend, G);
+  // Only the low byte of the reciprocal is useful in LE because it will get
+  // multiplied by another byte and then immediately masked by 0xFF.
+  C.Reciprocal = Mu.trunc(BW + 1).reverseBits().zext(ClmulBW).getLoBits(8);
+  // shl(1) here enables an optimization where two shifts then XOR can be
+  // replaced with an XOR then one shift. Normally one of the shifts would be by
+  // 7 and the other by 8, but by shifting here the shift by 7 becomes another
+  // shift by 8.
----------------
artagnon wrote:

The correct reason for shl(1) is because AArch64 CRC operates on bit-reflected data. The ASL code from the [manual](https://developer.arm.com/documentation/111183/2025-09_ASL1/Base-Instructions/CRC32--CRC32-?lang=en) specifies this:

```
bits(32)   acc  = X[n];    // accumulator
bits(size) val  = X[m];    // input value
bits(32)   poly = 0x04C11DB7;

bits(32+size) tempacc = BitReverse(acc):Zeros(size);
bits(size+32) tempval = BitReverse(val):Zeros(32);

// Poly32Mod2 on a bitstring does a polynomial Modulus over {0,1} operation
X[d] = BitReverse(Poly32Mod2(tempacc EOR tempval, poly));
```

However, llvm.clmul is not bit-reflected, and this shl(1) would be incorrect.

https://github.com/llvm/llvm-project/pull/203405


More information about the llvm-commits mailing list