[llvm] [X86][MC][AsmParser] Reject H-byte regs with VEX/EVEX-encoded 8-bit RR (NDD) (PR #160039)

[Phoebe Wang via llvm-commits]

================
@@ -4020,7 +4020,10 @@ bool X86AsmParser::validateInstruction(MCInst &Inst, const OperandVector &Ops) {
 
   // Check that we aren't mixing AH/BH/CH/DH with REX prefix. We only need to
   // check this with the legacy encoding, VEX/EVEX/XOP don't use REX.
-  if ((TSFlags & X86II::EncodingMask) == 0) {
+  const unsigned Enc = TSFlags & X86II::EncodingMask;
+  const bool UsesEvex = (Enc == X86II::EVEX);
+  const bool UsesRex2 = (ForcedOpcodePrefix == OpcodePrefix_REX2);
+  if (Enc == 0 || UsesEvex || UsesRex2) {
----------------
phoebewang wrote:

A new encoding would have its rules. We cannot forsee if it supports AH or not.

https://github.com/llvm/llvm-project/pull/160039