[all-commits] [llvm/llvm-project] aa21ce: [mlir] Do not set lastToken in AsmParser's resetTo...

[Jonas Rickert via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: aa21ce4a792c170074193c32e8ba8dd35e57c628
      https://github.com/llvm/llvm-project/commit/aa21ce4a792c170074193c32e8ba8dd35e57c628
  Author: Jonas Rickert <Jonas.Rickert at amd.com>
  Date:   2024-09-09 (Mon, 09 Sep 2024)

  Changed paths:
    M mlir/lib/AsmParser/Parser.h
    M mlir/unittests/Parser/CMakeLists.txt
    M mlir/unittests/Parser/ParserTest.cpp

  Log Message:
  -----------
  [mlir] Do not set lastToken in AsmParser's resetToken function and add a unit test for AsmParsers's locations (#105529)

This changes the function `resetToken` to not update `lastToken`.

The member `lastToken` is the last token that was consumed by the
parser.
Resetting the lexer position to a different position does not cause any
token to be consumed, so `lastToken` should not be updated.
Setting it to `curToken` can cause the scopeLoc.end location of
`OperationDefinition `to be off-by-one, pointing to the
first token after the operation.

An example for an operation for which the scopeLoc.end location was
wrong before is:
```
%0 = torch.vtensor.literal(dense_resource<__elided__> : tensor<768xbf16>) : !torch.vtensor<[768],bf16>
```
Here the scope end loc always pointed to the next token

This also adds a test for the Locations of `OperationDefinitions`.
Without the change to `resetToken` the test failes, with the scope end
location for `llvm.mlir.undef` pointing to the `func.return` in the next
line



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications