[lld] [ELF] Added `struct Token` and changed `next()` and `peek()` to return Token (PR #100180)

[Daniel Thornburgh via llvm-commits]

================
@@ -129,20 +146,19 @@ void ScriptLexer::tokenize(MemoryBufferRef mb) {
         return;
       }
 
-      vec.push_back(s.take_front(e + 1));
+      vec.push_back({Tok::Quote, s.take_front(e + 1)});
       s = s.substr(e + 1);
       continue;
     }
-
     // Some operators form separate tokens.
     if (s.starts_with("<<=") || s.starts_with(">>=")) {
----------------
mysterymath wrote:

There's a good deal of overlap between the logic that wraps these clauses and the logic in `getOperatorToken()`. Could we split `getOperatorToken()` into a family of functions that recognizes each of these classes of tokens, returning an optional token? We could probably also break out the quoted string recognization above into a similar function.

https://github.com/llvm/llvm-project/pull/100180