[llvm] [GlobalISel] Compact large match tables (PR #202642)
David Zbarsky via llvm-commits
llvm-commits at lists.llvm.org
Tue Jun 9 07:00:54 PDT 2026
https://github.com/dzbarsky created https://github.com/llvm/llvm-project/pull/202642
GlobalISel instruction selectors encode every failed-match destination as a 32-bit absolute offset and store common root operand indices as separate ULEB128 values. Large generated selectors repeat these fields enough to consume nearly a megabyte in all-target tools.
Compact tables of at least 64 KiB after generation. Encode forward failure destinations as 8-bit or 16-bit relative offsets when possible, fuse the compact forms with feature checks, and encode root operand indices 0 through 8 in the opcode. Keep the existing general encodings as fallbacks, so small tables and uncommon values are unchanged.
A fully stripped arm64 Release llc shrinks from 116,602,944 to 115,678,272 bytes, saving 924,672 bytes (0.793%). Generated match tables shrink by 918,123 bytes, led by 532,679 bytes for RISCV and 209,218 bytes for AMDGPU. The stripped downstream all-tools multicall binary shrinks from 142,874,088 to 142,015,464 bytes, saving 858,624 bytes (0.601%).
All 67 GlobalISel TableGen tests and all 17 TableGen unit tests pass, including new focused coverage for both relative-offset widths and the compact root operand forms. RISCV GlobalISel passes 448 tests with one unsupported, and AMDGPU GlobalISel passes all 1,060 tests. Baseline and candidate codegen is byte-identical across 88 successful RISCV cases, 368 successful AMDGPU cases, tramp3d-v4, and miniGMG.
CTMark RISCV GlobalISel user CPU changes by +0.22%; miniGMG improves by 1.26%, and the AMDGPU selection workload improves by 1.55%. RISCV TableGen improves slightly, while AMDGPU TableGen changes by +0.10%.
Work towards #202616
>From 8d2adb100d2259daf2feddc36329a32d30a87222 Mon Sep 17 00:00:00 2001
From: David Zbarsky <dzbarsky at gmail.com>
Date: Mon, 8 Jun 2026 16:51:35 -0400
Subject: [PATCH] [GlobalISel] Compact large match tables
GlobalISel instruction selectors encode every failed-match destination as a 32-bit absolute offset and store common root operand indices as separate ULEB128 values. Large generated selectors repeat these fields enough to consume nearly a megabyte in all-target tools.
Compact tables of at least 64 KiB after generation. Encode forward failure destinations as 8-bit or 16-bit relative offsets when possible, fuse the compact forms with feature checks, and encode root operand indices 0 through 8 in the opcode. Keep the existing general encodings as fallbacks, so small tables and uncommon values are unchanged.
A fully stripped arm64 Release llc shrinks from 116,602,944 to 115,678,272 bytes, saving 924,672 bytes (0.793%). Generated match tables shrink by 918,123 bytes, led by 532,679 bytes for RISCV and 209,218 bytes for AMDGPU. The stripped downstream all-tools multicall binary shrinks from 142,874,088 to 142,015,464 bytes, saving 858,624 bytes (0.601%).
All 67 GlobalISel TableGen tests and all 17 TableGen unit tests pass, including new focused coverage for both relative-offset widths and the compact root operand forms. RISCV GlobalISel passes 448 tests with one unsupported, and AMDGPU GlobalISel passes all 1,060 tests. Baseline and candidate codegen is byte-identical across 88 successful RISCV cases, 368 successful AMDGPU cases, tramp3d-v4, and miniGMG.
CTMark RISCV GlobalISel user CPU changes by +0.22%; miniGMG improves by 1.26%, and the AMDGPU selection workload improves by 1.55%. RISCV TableGen improves slightly, while AMDGPU TableGen changes by +0.10%.
---
.../CodeGen/GlobalISel/GIMatchTableExecutor.h | 40 ++++++
.../GlobalISel/GIMatchTableExecutorImpl.h | 87 +++++++++++--
llvm/unittests/TableGen/CMakeLists.txt | 1 +
.../TableGen/GlobalISelMatchTableTest.cpp | 54 ++++++++
.../GlobalISel/MatchTable/MatchTable.cpp | 122 +++++++++++++++++-
.../Common/GlobalISel/MatchTable/MatchTable.h | 22 +++-
.../Common/GlobalISel/MatchTable/Matchers.cpp | 4 +-
7 files changed, 308 insertions(+), 22 deletions(-)
create mode 100644 llvm/unittests/TableGen/GlobalISelMatchTableTest.cpp
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h b/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h
index 6e3ccf1923c40..1a53613383568 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h
@@ -91,11 +91,19 @@ enum {
/// requires an extra opcode and iteration in the interpreter on each
/// failed match.
GIM_Try,
+ /// GIM_Try with an 8-bit forward-relative OnFail.
+ GIM_Try8,
+ /// GIM_Try with a 16-bit forward-relative OnFail.
+ GIM_Try16,
/// GIM_Try only if the feature bits match.
/// - OnFail(4) - The MatchTable entry at which to resume if the match fails.
/// - Feature(2) - Expected features
GIM_Try_CheckFeatures,
+ /// GIM_Try_CheckFeatures with an 8-bit forward-relative OnFail.
+ GIM_Try_CheckFeatures8,
+ /// GIM_Try_CheckFeatures with a 16-bit forward-relative OnFail.
+ GIM_Try_CheckFeatures16,
/// Switch over the opcode on the specified instruction
/// - InsnID(ULEB128) - Instruction ID
@@ -250,6 +258,16 @@ enum {
GIM_CheckType,
/// GIM_CheckType but InsnID is omitted and defaults to zero.
GIM_RootCheckType,
+ /// GIM_RootCheckType with OpIdx encoded in the opcode.
+ GIM_RootCheckType0,
+ GIM_RootCheckType1,
+ GIM_RootCheckType2,
+ GIM_RootCheckType3,
+ GIM_RootCheckType4,
+ GIM_RootCheckType5,
+ GIM_RootCheckType6,
+ GIM_RootCheckType7,
+ GIM_RootCheckType8,
/// Check the type of a pointer to any address space.
/// - InsnID(ULEB128) - Instruction ID
@@ -264,6 +282,16 @@ enum {
GIM_CheckRegBankForClass,
/// GIM_CheckRegBankForClass but InsnID is omitted and defaults to zero.
GIM_RootCheckRegBankForClass,
+ /// GIM_RootCheckRegBankForClass with OpIdx encoded in the opcode.
+ GIM_RootCheckRegBankForClass0,
+ GIM_RootCheckRegBankForClass1,
+ GIM_RootCheckRegBankForClass2,
+ GIM_RootCheckRegBankForClass3,
+ GIM_RootCheckRegBankForClass4,
+ GIM_RootCheckRegBankForClass5,
+ GIM_RootCheckRegBankForClass6,
+ GIM_RootCheckRegBankForClass7,
+ GIM_RootCheckRegBankForClass8,
/// Check the operand matches a complex predicate
/// - InsnID(ULEB128) - Instruction ID
@@ -391,6 +419,16 @@ enum {
GIR_Copy,
/// GIR_Copy but with both New/OldInsnIDs omitted and defaulting to zero.
GIR_RootToRootCopy,
+ /// GIR_RootToRootCopy with OpIdx encoded in the opcode.
+ GIR_RootToRootCopy0,
+ GIR_RootToRootCopy1,
+ GIR_RootToRootCopy2,
+ GIR_RootToRootCopy3,
+ GIR_RootToRootCopy4,
+ GIR_RootToRootCopy5,
+ GIR_RootToRootCopy6,
+ GIR_RootToRootCopy7,
+ GIR_RootToRootCopy8,
/// Copies all operand starting from OpIdx in OldInsnID into the new
/// instruction NewInsnID.
@@ -597,6 +635,8 @@ enum {
/// Keeping track of the number of the GI opcodes. Must be the last entry.
GIU_NumOpcodes,
};
+static_assert(GIU_NumOpcodes <= 256,
+ "GlobalISel opcodes must fit in a match table byte");
/// Provides the logic to execute GlobalISel match tables, which are used by the
/// instruction selector and instruction combiners as their engine to match and
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h b/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h
index 70ee75108ffb8..1d74910e8d784 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h
@@ -149,16 +149,39 @@ bool GIMatchTableExecutor::executeMatchTable(
assert(CurrentIdx != ~0u && "Invalid MatchTable index");
uint8_t MatcherOpcode = MatchTable[CurrentIdx++];
switch (MatcherOpcode) {
- case GIM_Try: {
+ case GIM_Try:
+ case GIM_Try8:
+ case GIM_Try16: {
DEBUG_WITH_TYPE(TgtExecutor::getName(),
dbgs() << CurrentIdx << ": Begin try-block\n");
- OnFailResumeAt.push_back(readU32());
+ unsigned OnFail;
+ if (MatcherOpcode == GIM_Try) {
+ OnFail = readU32();
+ } else if (MatcherOpcode == GIM_Try8) {
+ OnFail = MatchTable[CurrentIdx++];
+ OnFail += CurrentIdx;
+ } else {
+ OnFail = readU16();
+ OnFail += CurrentIdx;
+ }
+ OnFailResumeAt.push_back(OnFail);
break;
}
- case GIM_Try_CheckFeatures: {
+ case GIM_Try_CheckFeatures:
+ case GIM_Try_CheckFeatures8:
+ case GIM_Try_CheckFeatures16: {
// This is optimized so that if the feature is not present, we don't even
// modify OnFailResumeAt. Instead we directly jump to OnFail.
- unsigned OnFail = readU32();
+ unsigned OnFail;
+ if (MatcherOpcode == GIM_Try_CheckFeatures) {
+ OnFail = readU32();
+ } else if (MatcherOpcode == GIM_Try_CheckFeatures8) {
+ OnFail = MatchTable[CurrentIdx++];
+ OnFail += CurrentIdx;
+ } else {
+ OnFail = readU16();
+ OnFail += CurrentIdx;
+ }
uint16_t ExpectedBitsetID = readU16();
DEBUG_WITH_TYPE(TgtExecutor::getName(),
dbgs() << CurrentIdx
@@ -741,10 +764,23 @@ bool GIMatchTableExecutor::executeMatchTable(
break;
}
+ case GIM_RootCheckType0:
+ case GIM_RootCheckType1:
+ case GIM_RootCheckType2:
+ case GIM_RootCheckType3:
+ case GIM_RootCheckType4:
+ case GIM_RootCheckType5:
+ case GIM_RootCheckType6:
+ case GIM_RootCheckType7:
+ case GIM_RootCheckType8:
case GIM_RootCheckType:
case GIM_CheckType: {
- uint64_t InsnID = (MatcherOpcode == GIM_RootCheckType) ? 0 : readULEB();
- uint64_t OpIdx = readULEB();
+ bool IsRoot = MatcherOpcode >= GIM_RootCheckType &&
+ MatcherOpcode <= GIM_RootCheckType8;
+ uint64_t InsnID = IsRoot ? 0 : readULEB();
+ uint64_t OpIdx = MatcherOpcode >= GIM_RootCheckType0
+ ? MatcherOpcode - GIM_RootCheckType0
+ : readULEB();
int TypeID = readS8();
DEBUG_WITH_TYPE(TgtExecutor::getName(),
dbgs() << CurrentIdx << ": GIM_CheckType(MIs[" << InsnID
@@ -823,11 +859,23 @@ bool GIMatchTableExecutor::executeMatchTable(
break;
}
+ case GIM_RootCheckRegBankForClass0:
+ case GIM_RootCheckRegBankForClass1:
+ case GIM_RootCheckRegBankForClass2:
+ case GIM_RootCheckRegBankForClass3:
+ case GIM_RootCheckRegBankForClass4:
+ case GIM_RootCheckRegBankForClass5:
+ case GIM_RootCheckRegBankForClass6:
+ case GIM_RootCheckRegBankForClass7:
+ case GIM_RootCheckRegBankForClass8:
case GIM_RootCheckRegBankForClass:
case GIM_CheckRegBankForClass: {
- uint64_t InsnID =
- (MatcherOpcode == GIM_RootCheckRegBankForClass) ? 0 : readULEB();
- uint64_t OpIdx = readULEB();
+ bool IsRoot = MatcherOpcode >= GIM_RootCheckRegBankForClass &&
+ MatcherOpcode <= GIM_RootCheckRegBankForClass8;
+ uint64_t InsnID = IsRoot ? 0 : readULEB();
+ uint64_t OpIdx = MatcherOpcode >= GIM_RootCheckRegBankForClass0
+ ? MatcherOpcode - GIM_RootCheckRegBankForClass0
+ : readULEB();
uint16_t RCEnum = readU16();
DEBUG_WITH_TYPE(TgtExecutor::getName(),
dbgs() << CurrentIdx << ": GIM_CheckRegBankForClass(MIs["
@@ -1123,13 +1171,24 @@ bool GIMatchTableExecutor::executeMatchTable(
break;
}
+ case GIR_RootToRootCopy0:
+ case GIR_RootToRootCopy1:
+ case GIR_RootToRootCopy2:
+ case GIR_RootToRootCopy3:
+ case GIR_RootToRootCopy4:
+ case GIR_RootToRootCopy5:
+ case GIR_RootToRootCopy6:
+ case GIR_RootToRootCopy7:
+ case GIR_RootToRootCopy8:
case GIR_RootToRootCopy:
case GIR_Copy: {
- uint64_t NewInsnID =
- (MatcherOpcode == GIR_RootToRootCopy) ? 0 : readULEB();
- uint64_t OldInsnID =
- (MatcherOpcode == GIR_RootToRootCopy) ? 0 : readULEB();
- uint64_t OpIdx = readULEB();
+ bool IsRoot = MatcherOpcode >= GIR_RootToRootCopy &&
+ MatcherOpcode <= GIR_RootToRootCopy8;
+ uint64_t NewInsnID = IsRoot ? 0 : readULEB();
+ uint64_t OldInsnID = IsRoot ? 0 : readULEB();
+ uint64_t OpIdx = MatcherOpcode >= GIR_RootToRootCopy0
+ ? MatcherOpcode - GIR_RootToRootCopy0
+ : readULEB();
assert(OutMIs[NewInsnID] && "Attempted to add to undefined instruction");
OutMIs[NewInsnID].add(State.MIs[OldInsnID]->getOperand(OpIdx));
DEBUG_WITH_TYPE(TgtExecutor::getName(),
diff --git a/llvm/unittests/TableGen/CMakeLists.txt b/llvm/unittests/TableGen/CMakeLists.txt
index 854f6c0f9b162..343f5696caa24 100644
--- a/llvm/unittests/TableGen/CMakeLists.txt
+++ b/llvm/unittests/TableGen/CMakeLists.txt
@@ -12,6 +12,7 @@ add_public_tablegen_target(AutomataTestTableGen)
add_llvm_unittest(TableGenTests
AutomataTest.cpp
CodeExpanderTest.cpp
+ GlobalISelMatchTableTest.cpp
ParserEntryPointTest.cpp
DISABLE_LLVM_LINK_LLVM_DYLIB
diff --git a/llvm/unittests/TableGen/GlobalISelMatchTableTest.cpp b/llvm/unittests/TableGen/GlobalISelMatchTableTest.cpp
new file mode 100644
index 0000000000000..4ba203551bc61
--- /dev/null
+++ b/llvm/unittests/TableGen/GlobalISelMatchTableTest.cpp
@@ -0,0 +1,54 @@
+//===- GlobalISelMatchTableTest.cpp --------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "Common/GlobalISel/MatchTable/MatchTable.h"
+#include "llvm/Support/raw_ostream.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+using namespace llvm::gi;
+
+TEST(GlobalISelMatchTableTest, CompactLargeTable) {
+ MatchTable Table(/*WithCoverage=*/false, /*IsCombinerTable=*/false);
+
+ // Cross the production compaction threshold before adding representative
+ // match records. Keeping the padding before the jumps does not affect their
+ // relative distances.
+ for (unsigned I = 0; I != 8192; ++I)
+ Table << MatchTable::IntValue(8, 0);
+
+ unsigned FarLabel = Table.allocateLabelID();
+ Table << MatchTable::Opcode("GIM_Try") << MatchTable::JumpTarget(FarLabel)
+ << MatchTable::Opcode("GIM_RootCheckType")
+ << MatchTable::ULEB128Value(2)
+ << MatchTable::Opcode("GIM_RootCheckRegBankForClass")
+ << MatchTable::ULEB128Value(3)
+ << MatchTable::Opcode("GIR_RootToRootCopy")
+ << MatchTable::ULEB128Value(8);
+ for (unsigned I = 0; I != 300; ++I)
+ Table << MatchTable::IntValue(1, 0);
+ Table << MatchTable::Label(FarLabel);
+
+ unsigned NearLabel = Table.allocateLabelID();
+ Table << MatchTable::Opcode("GIM_Try_CheckFeatures")
+ << MatchTable::JumpTarget(NearLabel)
+ << MatchTable::NamedValue(2, "FeatureBitset")
+ << MatchTable::IntValue(1, 0) << MatchTable::Label(NearLabel);
+
+ Table.compact();
+
+ std::string Output;
+ raw_string_ostream OS(Output);
+ Table.emitDeclaration(OS);
+
+ EXPECT_NE(Output.find("GIM_Try16"), std::string::npos);
+ EXPECT_NE(Output.find("GIM_Try_CheckFeatures8"), std::string::npos);
+ EXPECT_NE(Output.find("GIM_RootCheckType2"), std::string::npos);
+ EXPECT_NE(Output.find("GIM_RootCheckRegBankForClass3"), std::string::npos);
+ EXPECT_NE(Output.find("GIR_RootToRootCopy8"), std::string::npos);
+}
diff --git a/llvm/utils/TableGen/Common/GlobalISel/MatchTable/MatchTable.cpp b/llvm/utils/TableGen/Common/GlobalISel/MatchTable/MatchTable.cpp
index 7bebeee0db7df..3f0251fafc55c 100644
--- a/llvm/utils/TableGen/Common/GlobalISel/MatchTable/MatchTable.cpp
+++ b/llvm/utils/TableGen/Common/GlobalISel/MatchTable/MatchTable.cpp
@@ -7,6 +7,7 @@
//===----------------------------------------------------------------------===//
#include "MatchTable.h"
+#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/Twine.h"
#include "llvm/Support/LEB128.h"
#include "llvm/Support/ScopedPrinter.h"
@@ -20,6 +21,10 @@ namespace gi {
// GIMT_Encode2/4/8
constexpr StringLiteral EncodeMacroName = "GIMT_Encode";
+// Avoid adding specialized bytecode to small tables where the linked size
+// reduction is negligible.
+constexpr unsigned MinMatchTableSizeForCompaction = 64 * 1024;
+
//===- Helpers ------------------------------------------------------------===//
void emitEncodingMacrosDef(raw_ostream &OS) {
@@ -73,7 +78,8 @@ static std::string getEncodedEmitStr(StringRef NamedValue, unsigned NumBytes) {
//===- MatchTableRecord ---------------------------------------------------===//
void MatchTableRecord::emit(raw_ostream &OS, bool LineBreakIsNextAfterThis,
- const MatchTable &Table) const {
+ const MatchTable &Table,
+ unsigned CurrentIndex) const {
bool UseLineComment =
LineBreakIsNextAfterThis || (Flags & MTRF_LineBreakFollows);
if (Flags & (MTRF_JumpTarget | MTRF_CommaFollows))
@@ -96,9 +102,17 @@ void MatchTableRecord::emit(raw_ostream &OS, bool LineBreakIsNextAfterThis,
if (Flags & MTRF_JumpTarget) {
if (Flags & MTRF_Comment)
OS << " ";
- // TODO: Could encode this AOT to speed up build of generated file
- OS << getEncodedEmitStr(llvm::to_string(Table.getLabelIndex(LabelID)),
- NumElements);
+ unsigned Target = Table.getLabelIndex(LabelID);
+ if (Flags & MTRF_RelativeJumpTarget) {
+ assert(Target >= CurrentIndex + NumElements &&
+ "Relative jumps must be forward");
+ Target -= CurrentIndex + NumElements;
+ }
+ // TODO: Could encode this AOT to speed up build of generated file.
+ if (NumElements == 1)
+ OS << Target;
+ else
+ OS << getEncodedEmitStr(llvm::to_string(Target), NumElements);
}
if (Flags & MTRF_CommaFollows) {
@@ -204,7 +218,7 @@ void MatchTable::emitDeclaration(raw_ostream &OS) const {
static constexpr unsigned BaseIndent = 4;
unsigned Indentation = 0;
OS << " constexpr static uint8_t MatchTable" << ID << "[] = {";
- LineBreak.emit(OS, true, *this);
+ LineBreak.emit(OS, true, *this, 0);
// We want to display the table index of each line in a consistent
// manner. It has to appear as a column on the left side of the table.
@@ -236,7 +250,7 @@ void MatchTable::emitDeclaration(raw_ostream &OS) const {
if (I->Flags & MatchTableRecord::MTRF_Indent)
Indentation += 2;
- I->emit(OS, LineBreakIsNext, *this);
+ I->emit(OS, LineBreakIsNext, *this, CurIndex);
if (I->Flags & MatchTableRecord::MTRF_LineBreakFollows)
BeginLine();
@@ -249,5 +263,101 @@ void MatchTable::emitDeclaration(raw_ostream &OS) const {
OS << "}; // Size: " << CurrentSize << " bytes\n";
}
+void MatchTable::rebuildLabelMap() {
+ LabelMap.clear();
+ CurrentSize = 0;
+ for (const MatchTableRecord &Record : Contents) {
+ if (Record.Flags & MatchTableRecord::MTRF_Label)
+ defineLabel(Record.LabelID);
+ CurrentSize += Record.size();
+ }
+}
+
+void MatchTable::compactFailureTargets() {
+ if (CurrentSize < MinMatchTableSizeForCompaction)
+ return;
+
+ std::vector<unsigned> RecordOffsets;
+ RecordOffsets.reserve(Contents.size());
+ unsigned Offset = 0;
+ for (const MatchTableRecord &Record : Contents) {
+ RecordOffsets.push_back(Offset);
+ Offset += Record.size();
+ }
+
+ for (unsigned I = 0, E = Contents.size(); I != E; ++I) {
+ MatchTableRecord &Opcode = Contents[I];
+ StringRef OpcodeName = Opcode.EmitStr;
+ if (OpcodeName != "GIM_Try" && OpcodeName != "GIM_Try_CheckFeatures")
+ continue;
+
+ unsigned JumpRecord = I + 1;
+ while (JumpRecord != E && Contents[JumpRecord].size() == 0)
+ ++JumpRecord;
+ assert(JumpRecord != E &&
+ (Contents[JumpRecord].Flags & MatchTableRecord::MTRF_JumpTarget));
+
+ MatchTableRecord &Jump = Contents[JumpRecord];
+ unsigned Target = getLabelIndex(Jump.LabelID);
+ unsigned JumpOffset = RecordOffsets[JumpRecord];
+ assert(Target > JumpOffset && "GIM_Try targets must be forward");
+
+ unsigned NumBytes;
+ StringRef Suffix;
+ unsigned Distance = Target - JumpOffset;
+ if (Distance <= 256) {
+ NumBytes = 1;
+ Suffix = "8";
+ } else if (Distance <= 65537) {
+ NumBytes = 2;
+ Suffix = "16";
+ } else {
+ continue;
+ }
+
+ Opcode.EmitStr += Suffix;
+ Jump.makeRelativeJumpTarget(NumBytes);
+ }
+
+ rebuildLabelMap();
+}
+
+void MatchTable::compactRootOperandIndices() {
+ if (CurrentSize < MinMatchTableSizeForCompaction)
+ return;
+
+ static constexpr StringLiteral RootOperandOpcodes[] = {
+ "GIM_RootCheckType", "GIM_RootCheckRegBankForClass",
+ "GIR_RootToRootCopy"};
+
+ for (unsigned I = 0, E = Contents.size(); I != E; ++I) {
+ MatchTableRecord &Opcode = Contents[I];
+ if (!is_contained(RootOperandOpcodes, Opcode.EmitStr))
+ continue;
+
+ unsigned OperandRecord = I + 1;
+ while (OperandRecord != E && Contents[OperandRecord].size() == 0)
+ ++OperandRecord;
+ assert(OperandRecord != E && "Missing root operand index");
+
+ MatchTableRecord &Operand = Contents[OperandRecord];
+ unsigned OperandIndex;
+ if (Operand.size() != 1 ||
+ StringRef(Operand.EmitStr).getAsInteger(10, OperandIndex) ||
+ OperandIndex > 8)
+ continue;
+
+ Opcode.EmitStr += llvm::to_string(OperandIndex);
+ Operand.clear();
+ }
+
+ rebuildLabelMap();
+}
+
+void MatchTable::compact() {
+ compactFailureTargets();
+ compactRootOperandIndices();
+}
+
} // namespace gi
} // namespace llvm
diff --git a/llvm/utils/TableGen/Common/GlobalISel/MatchTable/MatchTable.h b/llvm/utils/TableGen/Common/GlobalISel/MatchTable/MatchTable.h
index a7ac109fb99c0..be840420d3b31 100644
--- a/llvm/utils/TableGen/Common/GlobalISel/MatchTable/MatchTable.h
+++ b/llvm/utils/TableGen/Common/GlobalISel/MatchTable/MatchTable.h
@@ -63,6 +63,8 @@ struct MatchTableRecord {
/// Causes the formatter to not use encoding macros to emit this multi-byte
/// value.
MTRF_PreEncoded = 0x80,
+ /// Causes a jump target to be emitted relative to the end of this record.
+ MTRF_RelativeJumpTarget = 0x100,
};
/// When MTRF_Label or MTRF_JumpTarget is used, indicates a label id to
@@ -99,8 +101,21 @@ struct MatchTableRecord {
NumElements = 0;
}
+ void makeRelativeJumpTarget(unsigned NumBytes) {
+ assert(Flags & MTRF_JumpTarget);
+ assert((NumBytes == 1 || NumBytes == 2) && "Unsupported jump size");
+ Flags |= MTRF_RelativeJumpTarget;
+ NumElements = NumBytes;
+ }
+
+ void clear() {
+ EmitStr.clear();
+ NumElements = 0;
+ Flags = MTRF_None;
+ }
+
void emit(raw_ostream &OS, bool LineBreakNextAfterThis,
- const MatchTable &Table) const;
+ const MatchTable &Table, unsigned CurrentIndex) const;
unsigned size() const { return NumElements; }
};
@@ -124,6 +139,10 @@ class MatchTable {
/// Whether this table is for the GISel combiner.
bool IsCombinerTable;
+ void compactFailureTargets();
+ void compactRootOperandIndices();
+ void rebuildLabelMap();
+
public:
static MatchTableRecord LineBreak;
static MatchTableRecord Comment(StringRef Comment);
@@ -142,6 +161,7 @@ class MatchTable {
bool isWithCoverage() const { return IsWithCoverage; }
bool isCombiner() const { return IsCombinerTable; }
+ void compact();
void push_back(const MatchTableRecord &Value) {
if (Value.Flags & MatchTableRecord::MTRF_Label)
diff --git a/llvm/utils/TableGen/Common/GlobalISel/MatchTable/Matchers.cpp b/llvm/utils/TableGen/Common/GlobalISel/MatchTable/Matchers.cpp
index 7f22cbfc7f58d..1ba100747348f 100644
--- a/llvm/utils/TableGen/Common/GlobalISel/MatchTable/Matchers.cpp
+++ b/llvm/utils/TableGen/Common/GlobalISel/MatchTable/Matchers.cpp
@@ -141,7 +141,9 @@ MatchTable llvm::gi::buildMatchTable(ArrayRef<Matcher *> Rules,
for (Matcher *Rule : Rules)
Rule->emit(Table);
- return Table << MatchTable::Opcode("GIM_Reject") << MatchTable::LineBreak;
+ Table << MatchTable::Opcode("GIM_Reject") << MatchTable::LineBreak;
+ Table.compact();
+ return Table;
}
template <class Range> static bool matchersRecordOperand(Range &&R) {
More information about the llvm-commits
mailing list