Allow peeking forward several tokens with MCAsmLexer

[Dylan McKay via llvm-commits]

Ready to be committed.

On Mon, Aug 17, 2015 at 3:30 AM, Dylan McKay <dylanmckay34 at gmail.com> wrote:

> You're right, it is now fixed.
>
> On Mon, Aug 17, 2015 at 3:02 AM, Benjamin Kramer <benny.kra at gmail.com>
> wrote:
>
>>
>> > On 16.08.2015, at 16:53, Dylan McKay <dylanmckay34 at gmail.com> wrote:
>> >
>> > I fixed everything you mentioned Benjamin - thanks for the review!
>> >
>> > Attached is the new patch.
>>
>> You don't need the size_t Count argument anymore, the MutableArrayRef
>> already carries the size.
>>
>> - Ben
>>
>> >
>> > On Sun, Aug 16, 2015 at 10:57 PM, Benjamin Kramer <benny.kra at gmail.com>
>> wrote:
>> >
>> > > On 15.08.2015, at 16:23, Dylan McKay via llvm-commits <
>> llvm-commits at lists.llvm.org> wrote:
>> > >
>> > > From b7094fa637ffd59b10776e0a4bbf7f8c04235306 Mon Sep 17 00:00:00 2001
>> > > From: Dylan McKay <dylanmckay34 at gmail.com>
>> > > Date: Sun, 16 Aug 2015 02:09:05 +1200
>> > > Subject: [PATCH] Extend MCAsmLexer so that it can peek forward
>> several tokens
>> > >
>> > > This commit adds a virtual `peekTokens()` function to `MCAsmLexer`
>> > > which can peek forward an arbitrary number of tokens.
>> > >
>> > > It also makes the `peekTok()` method call `peekTokens()` method, but
>> > > only requesting one token.
>> > >
>> > > The idea is to better support targets which more more ambiguous
>> > > assembly syntaxes.
>> > > ---
>> > >  include/llvm/MC/MCParser/AsmLexer.h   |  3 ++-
>> > >  include/llvm/MC/MCParser/MCAsmLexer.h | 13 ++++++++++++-
>> > >  lib/MC/MCParser/AsmLexer.cpp          | 16 +++++++++++++---
>> > >  3 files changed, 27 insertions(+), 5 deletions(-)
>> > >
>> > > diff --git a/include/llvm/MC/MCParser/AsmLexer.h
>> b/include/llvm/MC/MCParser/AsmLexer.h
>> > > index 62d39b2..77023e4 100644
>> > > --- a/include/llvm/MC/MCParser/AsmLexer.h
>> > > +++ b/include/llvm/MC/MCParser/AsmLexer.h
>> > > @@ -47,7 +47,8 @@ public:
>> > >    StringRef LexUntilEndOfStatement() override;
>> > >    StringRef LexUntilEndOfLine();
>> > >
>> > > -  const AsmToken peekTok(bool ShouldSkipSpace = true) override;
>> > > +  size_t peekTokens(AsmToken *Buf, size_t Size,
>> > > +                    bool ShouldSkipSpace = true) override;
>> >
>> > This could be a MutableArrayRef<AsmToken> instead of a pointer+size
>> pair.
>> >
>> > >
>> > >    bool isAtStartOfComment(const char *Ptr);
>> > >    bool isAtStatementSeparator(const char *Ptr);
>> > > diff --git a/include/llvm/MC/MCParser/MCAsmLexer.h
>> b/include/llvm/MC/MCParser/MCAsmLexer.h
>> > > index 71f15b3..fb9d32d 100644
>> > > --- a/include/llvm/MC/MCParser/MCAsmLexer.h
>> > > +++ b/include/llvm/MC/MCParser/MCAsmLexer.h
>> > > @@ -162,7 +162,18 @@ public:
>> > >    }
>> > >
>> > >    /// Look ahead at the next token to be lexed.
>> > > -  virtual const AsmToken peekTok(bool ShouldSkipSpace = true) = 0;
>> > > +  const AsmToken peekTok(bool ShouldSkipSpace = true) {
>> > > +    AsmToken Buf;
>> > > +
>> > > +    size_t ReadCount = peekTokens(&Buf, 1, ShouldSkipSpace);
>> > > +    assert(ReadCount == 1);
>> >
>> > Add (void)ReadCount; here to avoid unused variable warnings when
>> assertions are disabled.
>> >
>> > > +
>> > > +    return Buf;
>> > > +  }
>> > > +
>> > > +  /// Look ahead an arbitrary number of tokens.
>> > > +  virtual size_t peekTokens(AsmToken *Buf, size_t Size,
>> > > +                            bool ShouldSkipSpace = true) = 0;
>> > >
>> > >    /// Get the current error location
>> > >    const SMLoc &getErrLoc() {
>> > > diff --git a/lib/MC/MCParser/AsmLexer.cpp
>> b/lib/MC/MCParser/AsmLexer.cpp
>> > > index b983d99..fcedb42 100644
>> > > --- a/lib/MC/MCParser/AsmLexer.cpp
>> > > +++ b/lib/MC/MCParser/AsmLexer.cpp
>> > > @@ -436,7 +436,8 @@ StringRef AsmLexer::LexUntilEndOfLine() {
>> > >    return StringRef(TokStart, CurPtr-TokStart);
>> > >  }
>> > >
>> > > -const AsmToken AsmLexer::peekTok(bool ShouldSkipSpace) {
>> > > +size_t AsmLexer::peekTokens(AsmToken *Buf, size_t Size,
>> > > +                         bool ShouldSkipSpace) {
>> > >    const char *SavedTokStart = TokStart;
>> > >    const char *SavedCurPtr = CurPtr;
>> > >    bool SavedAtStartOfLine = isAtStartOfLine;
>> > > @@ -446,8 +447,17 @@ const AsmToken AsmLexer::peekTok(bool
>> ShouldSkipSpace) {
>> > >    SMLoc SavedErrLoc = getErrLoc();
>> > >
>> > >    SkipSpace = ShouldSkipSpace;
>> > > -  AsmToken Token = LexToken();
>> > >
>> > > +  size_t ReadCount;
>> > > +  for(ReadCount = 0; ReadCount<Size; ++ReadCount) {
>> > > +    AsmToken Token = LexToken();
>> > > +
>> > > +    Buf[ReadCount] = Token;
>> > > +
>> > > +    if(Token.is(AsmToken::Eof))
>> > > +        break;
>> > > +
>> > > +  }
>> >
>> > This hunk has funky identation. clang-format?
>> >
>> > Patch looks good when my comments are fixed.
>> >
>> > - Ben
>> >
>> > >    SetError(SavedErrLoc, SavedErr);
>> > >
>> > >    SkipSpace = SavedSkipSpace;
>> > > @@ -455,7 +465,7 @@ const AsmToken AsmLexer::peekTok(bool
>> ShouldSkipSpace) {
>> > >    CurPtr = SavedCurPtr;
>> > >    TokStart = SavedTokStart;
>> > >
>> > > -  return Token;
>> > > +  return ReadCount;
>> > >  }
>> > >
>> > >  bool AsmLexer::isAtStartOfComment(const char *Ptr) {
>> > > --
>> > > 2.4.4
>> > >
>> >
>> >
>> > <0001-Extend-MCAsmLexer-so-that-it-can-peek-forward-severa.patch>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150818/c134f527/attachment.html>