[llvm] [TableGen] Remove explicit recursion in LexToken (PR #143697)

[Jay Foad via llvm-commits]

https://github.com/jayfoad created https://github.com/llvm/llvm-project/pull/143697

When profiling a Release+Asserts build of llvm-tblgen I noticed that it
was recursing hundreds of times to lex a sequence of hundreds of space
characters.


>From b08e3994d93440791b3733f073dd0e1869d42955 Mon Sep 17 00:00:00 2001
From: Jay Foad <jay.foad at amd.com>
Date: Wed, 11 Jun 2025 13:56:04 +0100
Subject: [PATCH] [TableGen] Remove explicit recursion in LexToken

When profiling a Release+Asserts build of llvm-tblgen I noticed that it
was recursing hundreds of times to lex a sequence of hundreds of space
characters.
---
 llvm/lib/TableGen/TGLexer.cpp | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/llvm/lib/TableGen/TGLexer.cpp b/llvm/lib/TableGen/TGLexer.cpp
index 46487cba9453e..5a3f86372beca 100644
--- a/llvm/lib/TableGen/TGLexer.cpp
+++ b/llvm/lib/TableGen/TGLexer.cpp
@@ -174,6 +174,7 @@ int TGLexer::peekNextChar(int Index) const {
 }
 
 tgtok::TokKind TGLexer::LexToken(bool FileOrLineStart) {
+restart:
   TokStart = CurPtr;
   // This always consumes at least one character.
   int CurChar = getNextChar();
@@ -188,12 +189,12 @@ tgtok::TokKind TGLexer::LexToken(bool FileOrLineStart) {
     return ReturnError(TokStart, "unexpected character");
   case EOF:
     // Lex next token, if we just left an include file.
-    // Note that leaving an include file means that the next
-    // symbol is located at the end of the 'include "..."'
-    // construct, so LexToken() is called with default
-    // false parameter.
-    if (processEOF())
-      return LexToken();
+    if (processEOF()) {
+      // Leaving an include file means that the next symbol is located at the
+      // end of the 'include "..."'  construct.
+      FileOrLineStart = false;
+      goto restart;
+    }
 
     // Return EOF denoting the end of lexing.
     return tgtok::Eof;
@@ -238,10 +239,11 @@ tgtok::TokKind TGLexer::LexToken(bool FileOrLineStart) {
   case ' ':
   case '\t':
     // Ignore whitespace.
-    return LexToken(FileOrLineStart);
+    goto restart;
   case '\n':
     // Ignore whitespace, and identify the new line.
-    return LexToken(true);
+    FileOrLineStart = true;
+    goto restart;
   case '/':
     // If this is the start of a // comment, skip until the end of the line or
     // the end of the buffer.
@@ -252,7 +254,7 @@ tgtok::TokKind TGLexer::LexToken(bool FileOrLineStart) {
         return tgtok::Error;
     } else // Otherwise, this is an error.
       return ReturnError(TokStart, "unexpected character");
-    return LexToken(FileOrLineStart);
+    goto restart;
   case '-': case '+':
   case '0': case '1': case '2': case '3': case '4': case '5': case '6':
   case '7': case '8': case '9': {