[PATCH] D59885: [Lex] Allow to consume tokens while preprocessing
Ilya Biryukov via Phabricator via cfe-commits
cfe-commits at lists.llvm.org
Thu May 2 08:17:07 PDT 2019
ilya-biryukov marked an inline comment as not done.
ilya-biryukov added inline comments.
================
Comment at: clang/lib/Lex/Preprocessor.cpp:956-957
--LexLevel;
+ if (OnToken)
+ OnToken(Result, Source);
}
----------------
ilya-biryukov wrote:
> rsmith wrote:
> > This seems like it's going to be extremely hard to use. If you just want the expanded token stream, then removing all the other calls to `OnToken` and only calling it here when `LexLevel == 0` will give you that.
> I've tried `LexLevel == 0 && IsNewToken` and it **almost** works.
> The only trouble left is dealing with the delayed parsing. E.g. I'm getting the callbacks for the same tokens multiple times in cases like parsing of method bodies.
>
> Any ideas on how to best tackle this?
>
> The example is:
> ```
> struct X {
> int method() { return 10; }
> int a = 10;
> };
> ```
>
> Seeing multiple callbacks for `{ return 10; }` and `= 10;`, want to see only the first one.
It almost works now, see the update revision.
Almost no tokens get reported by the callback twice now, except the following case.
Input:
```
struct X {
int method() { return method(); }
};
```
After the body of the class, we get a an extra callback for `)` token from one of the functions under `ParsedLexedMethodDef`.
This seems to be happening only for parenthesis.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D59885/new/
https://reviews.llvm.org/D59885
More information about the cfe-commits
mailing list