<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/99920>99920</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [ELF] A new Lexer for Linker Script
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            lld:ELF,
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
            yugier
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          yugier
      </td>
    </tr>
</table>

<pre>
    - [x] ~~delete peek2~~ https://github.com/llvm/llvm-project/pull/99790 
- [ ] delete consumeLabel
- [ ] add `struct Token {StringRef str; TokenKind kind;}; vector<StringRef> tokens;` => `vector<Token> token;` change next/peek to return Token
- [ ] add a consume overload to accept Token. The existing consume(StringRef) is be a special case that expects identifier/quoted_string.
- [ ] introduce state to remove some unquote and https://github.com/llvm/llvm-project/pull/84130 hack
- [ ] introduce state to replace inExpr
- [ ] fully stateful lexer; replace eager tokenize with on-demand lexing;with on-demand lexing, replace getColumnNumber (from Token) with Lexer::getColumnNumber
- [ ] replace mbs with an include stack (INCLUDE); replace getCurrentMB()

We want to replace current lexer with a stateful one to report more errors and get rid of many hacks in current lexer such as `ScriptLexer::maybeSplitExpr`. The list of items is the plan to update lexer gradually. 
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJycVE1v6zYQ_DX0ZRFDovylgw52HANF3XdoXtFjQZEriTVFquTSsXvIby8o2UlgvEPbiw1QM7M7O-SKEHRrESu23DHOr7HV6BnnbLmfiUid89V0NqudulZPwJa7C1vu4f1doUFCGBBP_P0dOqIhsGLL-IHxQ6upi_Vcup7xgzHn-9_T4N2fKInxwxCNYfxQlusyA5btWbYd5SHJ38SlsyH2eBQ1mkeIUArYKgvkoyT47k5oga13r-S1bX_FBgJ5VuymLz9rq-CkrWLFjq336fyMkpxnxfMHgxUvQAkdEmqVASv26Yytsg_wqPYBvOFkJ2yLYPEyGkM8ATnwSNHbqf6Pmhd3e-DO6I0TKrGElDjc_Mzhe4eAFx1I2_YOZ3zz2TIvQQeoEQSEAaUWBqQICNQJArwMKCmAVmhJN2O0h7-iI1R_hFFi_tiYtuSdihIhkCCcfPTujBBcjxDtSAdh1f9OfLPIiww6IU__qvhghETQ9uUy-EdCE425TuAmGjB4wTHzOwtFi36KSv-N8KapA2efFPbJgMGLti0rdj8-588fOi3SszOxt99iX6MHxjeNd_0tW15Oysep_JYV2wfCY9933b4OE1VY0FaaqEbr8pQq_PTt-fjb_oXx8qulpBy9R0u_7BjfpK-j-PT7O8KbsPR1cnJCT8O5VfscmbP3MTtP0DuPgN47H8aIWyTwWoFroBf2OoYWQNsH0RBlByKkh_IqvR7oyyR6ca3xdTCaxgBX2XSpjQ6UZDVhH9IVpg5hMMKmbuKgUvyTeOuFisKY6xxmqipUWZRihlW-5vlqvVwt81lXqVVdKKGUUsWiLhdysywamWeIfJ03WV7MdMUzvsjWnOdlvsjzebNYLGtViE2zKVe4UWyRYS-0mac7O3e-nekQIlZlWfJsZtICCrclaYxixfbleEh7kj8zzi2-wQi_bU5fjTe_jm1giyxZDZ-6pMmM-zYpLPewhUQfJwaN83DU9oQepkHOojfVf35oYy9hXK6p-3PF_wkAAP__5-_vjw">