Allow peeking forward several tokens with MCAsmLexer

[Benjamin Kramer via llvm-commits]

> On 17.08.2015, at 16:25, Dylan McKay <dylanmckay34 at gmail.com> wrote:
> 
> Ready to be committed.

Done in r245221, thanks!

- Ben

> On Mon, Aug 17, 2015 at 3:30 AM, Dylan McKay <dylanmckay34 at gmail.com> wrote:
> You're right, it is now fixed.
> 
> On Mon, Aug 17, 2015 at 3:02 AM, Benjamin Kramer <benny.kra at gmail.com> wrote:
> 
> > On 16.08.2015, at 16:53, Dylan McKay <dylanmckay34 at gmail.com> wrote:
> >
> > I fixed everything you mentioned Benjamin - thanks for the review!
> >
> > Attached is the new patch.
> 
> You don't need the size_t Count argument anymore, the MutableArrayRef already carries the size.
> 
> - Ben
> 
> >
> > On Sun, Aug 16, 2015 at 10:57 PM, Benjamin Kramer <benny.kra at gmail.com> wrote:
> >
> > > On 15.08.2015, at 16:23, Dylan McKay via llvm-commits <llvm-commits at lists.llvm.org> wrote:
> > >
> > > From b7094fa637ffd59b10776e0a4bbf7f8c04235306 Mon Sep 17 00:00:00 2001
> > > From: Dylan McKay <dylanmckay34 at gmail.com>
> > > Date: Sun, 16 Aug 2015 02:09:05 +1200
> > > Subject: [PATCH] Extend MCAsmLexer so that it can peek forward several tokens
> > >
> > > This commit adds a virtual `peekTokens()` function to `MCAsmLexer`
> > > which can peek forward an arbitrary number of tokens.
> > >
> > > It also makes the `peekTok()` method call `peekTokens()` method, but
> > > only requesting one token.
> > >
> > > The idea is to better support targets which more more ambiguous
> > > assembly syntaxes.
> > > ---
> > >  include/llvm/MC/MCParser/AsmLexer.h   |  3 ++-
> > >  include/llvm/MC/MCParser/MCAsmLexer.h | 13 ++++++++++++-
> > >  lib/MC/MCParser/AsmLexer.cpp          | 16 +++++++++++++---
> > >  3 files changed, 27 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/include/llvm/MC/MCParser/AsmLexer.h b/include/llvm/MC/MCParser/AsmLexer.h
> > > index 62d39b2..77023e4 100644
> > > --- a/include/llvm/MC/MCParser/AsmLexer.h
> > > +++ b/include/llvm/MC/MCParser/AsmLexer.h
> > > @@ -47,7 +47,8 @@ public:
> > >    StringRef LexUntilEndOfStatement() override;
> > >    StringRef LexUntilEndOfLine();
> > >
> > > -  const AsmToken peekTok(bool ShouldSkipSpace = true) override;
> > > +  size_t peekTokens(AsmToken *Buf, size_t Size,
> > > +                    bool ShouldSkipSpace = true) override;
> >
> > This could be a MutableArrayRef<AsmToken> instead of a pointer+size pair.
> >
> > >
> > >    bool isAtStartOfComment(const char *Ptr);
> > >    bool isAtStatementSeparator(const char *Ptr);
> > > diff --git a/include/llvm/MC/MCParser/MCAsmLexer.h b/include/llvm/MC/MCParser/MCAsmLexer.h
> > > index 71f15b3..fb9d32d 100644
> > > --- a/include/llvm/MC/MCParser/MCAsmLexer.h
> > > +++ b/include/llvm/MC/MCParser/MCAsmLexer.h
> > > @@ -162,7 +162,18 @@ public:
> > >    }
> > >
> > >    /// Look ahead at the next token to be lexed.
> > > -  virtual const AsmToken peekTok(bool ShouldSkipSpace = true) = 0;
> > > +  const AsmToken peekTok(bool ShouldSkipSpace = true) {
> > > +    AsmToken Buf;
> > > +
> > > +    size_t ReadCount = peekTokens(&Buf, 1, ShouldSkipSpace);
> > > +    assert(ReadCount == 1);
> >
> > Add (void)ReadCount; here to avoid unused variable warnings when assertions are disabled.
> >
> > > +
> > > +    return Buf;
> > > +  }
> > > +
> > > +  /// Look ahead an arbitrary number of tokens.
> > > +  virtual size_t peekTokens(AsmToken *Buf, size_t Size,
> > > +                            bool ShouldSkipSpace = true) = 0;
> > >
> > >    /// Get the current error location
> > >    const SMLoc &getErrLoc() {
> > > diff --git a/lib/MC/MCParser/AsmLexer.cpp b/lib/MC/MCParser/AsmLexer.cpp
> > > index b983d99..fcedb42 100644
> > > --- a/lib/MC/MCParser/AsmLexer.cpp
> > > +++ b/lib/MC/MCParser/AsmLexer.cpp
> > > @@ -436,7 +436,8 @@ StringRef AsmLexer::LexUntilEndOfLine() {
> > >    return StringRef(TokStart, CurPtr-TokStart);
> > >  }
> > >
> > > -const AsmToken AsmLexer::peekTok(bool ShouldSkipSpace) {
> > > +size_t AsmLexer::peekTokens(AsmToken *Buf, size_t Size,
> > > +                         bool ShouldSkipSpace) {
> > >    const char *SavedTokStart = TokStart;
> > >    const char *SavedCurPtr = CurPtr;
> > >    bool SavedAtStartOfLine = isAtStartOfLine;
> > > @@ -446,8 +447,17 @@ const AsmToken AsmLexer::peekTok(bool ShouldSkipSpace) {
> > >    SMLoc SavedErrLoc = getErrLoc();
> > >
> > >    SkipSpace = ShouldSkipSpace;
> > > -  AsmToken Token = LexToken();
> > >
> > > +  size_t ReadCount;
> > > +  for(ReadCount = 0; ReadCount<Size; ++ReadCount) {
> > > +    AsmToken Token = LexToken();
> > > +
> > > +    Buf[ReadCount] = Token;
> > > +
> > > +    if(Token.is(AsmToken::Eof))
> > > +        break;
> > > +
> > > +  }
> >
> > This hunk has funky identation. clang-format?
> >
> > Patch looks good when my comments are fixed.
> >
> > - Ben
> >
> > >    SetError(SavedErrLoc, SavedErr);
> > >
> > >    SkipSpace = SavedSkipSpace;
> > > @@ -455,7 +465,7 @@ const AsmToken AsmLexer::peekTok(bool ShouldSkipSpace) {
> > >    CurPtr = SavedCurPtr;
> > >    TokStart = SavedTokStart;
> > >
> > > -  return Token;
> > > +  return ReadCount;
> > >  }
> > >
> > >  bool AsmLexer::isAtStartOfComment(const char *Ptr) {
> > > --
> > > 2.4.4
> > >
> >
> >
> > <0001-Extend-MCAsmLexer-so-that-it-can-peek-forward-severa.patch>
> 
> 
>