[all-commits] [llvm/llvm-project] ea488b: [mlir] Allow for attaching external resources to ....

[River Riddle via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: ea488bd6e1f7bf52d6ec7a40c7116670f06e92a6
      https://github.com/llvm/llvm-project/commit/ea488bd6e1f7bf52d6ec7a40c7116670f06e92a6
  Author: River Riddle <riddleriver at gmail.com>
  Date:   2022-06-29 (Wed, 29 Jun 2022)

  Changed paths:
    M mlir/include/mlir/IR/AsmState.h
    M mlir/include/mlir/IR/OpAsmInterface.td
    M mlir/include/mlir/IR/OpImplementation.h
    M mlir/include/mlir/Parser/Parser.h
    M mlir/lib/IR/AsmPrinter.cpp
    M mlir/lib/Parser/AffineParser.cpp
    M mlir/lib/Parser/AsmParserImpl.h
    M mlir/lib/Parser/DialectSymbolParser.cpp
    M mlir/lib/Parser/Lexer.cpp
    M mlir/lib/Parser/Parser.cpp
    M mlir/lib/Parser/Parser.h
    M mlir/lib/Parser/ParserState.h
    M mlir/lib/Parser/Token.cpp
    M mlir/lib/Parser/TokenKinds.def
    M mlir/lib/Parser/TypeParser.cpp
    M mlir/test/IR/elements-attr-interface.mlir
    A mlir/test/IR/file-metadata-resources.mlir
    A mlir/test/IR/invalid-file-metadata.mlir
    M mlir/test/lib/Dialect/Test/TestAttrDefs.td
    M mlir/test/lib/Dialect/Test/TestAttributes.cpp
    M mlir/test/lib/Dialect/Test/TestAttributes.h
    M mlir/test/lib/Dialect/Test/TestDialect.cpp
    M mlir/test/lib/Dialect/Test/TestDialect.h
    M mlir/test/lib/Dialect/Test/TestDialect.td
    M mlir/unittests/CMakeLists.txt
    A mlir/unittests/Parser/CMakeLists.txt
    A mlir/unittests/Parser/ResourceTest.cpp

  Log Message:
  -----------
  [mlir] Allow for attaching external resources to .mlir files

This commit enables support for providing and processing external
resources within MLIR assembly formats. This is a mechanism with which
dialects, and external clients, may attach additional information when
printing IR without that information being encoded in the IR itself.
External resources are not uniqued within the MLIR context, are not
attached directly to any operation, and are solely intended to live and be
processed outside of the immediate IR. There are many potential uses of this
functionality, for example MLIR's pass crash reproducer could utilize this to
attach the pass resource executing when a crash occurs. Other types of
uses may be embedding large amounts of binary data, such as weights in ML
applications, that shouldn't be copied directly into the MLIR context, but
need to be kept adjacent to the IR.

External resources are encoded using a key-value pair nested within a
dictionary anchored by name either on a dialect, or an externally registered
entity. The key is an identifier used to disambiguate the data. The value
may be stored in various limited forms, but general encodings use a string
(human readable) or blob format (binary). Within the textual format, an
example may be of the form:

```mlir
{-#
  // The `dialect_resources` section within the file-level metadata
  // dictionary is used to contain any dialect resource entries.
  dialect_resources: {
    // Here is a dictionary anchored on "foo_dialect", which is a dialect
    // namespace.
    foo_dialect: {
      // `some_dialect_resource` is a key to be interpreted by the dialect,
      // and used to initialize/configure/etc.
      some_dialect_resource: "Some important resource value"
    }
  },
  // The `external_resources` section within the file-level metadata
  // dictionary is used to contain any non-dialect resource entries.
  external_resources: {
    // Here is a dictionary anchored on "mlir_reproducer", which is an
    // external entity representing MLIR's crash reproducer functionality.
    mlir_reproducer: {
      // `pipeline` is an entry that holds a crash reproducer pipeline
      // resource.
      pipeline: "func.func(canonicalize,cse)"
    }
  }
```

Differential Revision: https://reviews.llvm.org/D126446


  Commit: 361acbb3627240d049407a164b355b50086f6d79
      https://github.com/llvm/llvm-project/commit/361acbb3627240d049407a164b355b50086f6d79
  Author: River Riddle <riddleriver at gmail.com>
  Date:   2022-06-29 (Wed, 29 Jun 2022)

  Changed paths:
    M mlir/include/mlir/Pass/PassRegistry.h
    M mlir/lib/Pass/PassCrashRecovery.cpp
    M mlir/lib/Tools/mlir-opt/MlirOptMain.cpp
    M mlir/test/Pass/crash-recovery-dynamic-failure.mlir
    M mlir/test/Pass/crash-recovery.mlir
    M mlir/test/Pass/run-reproducer.mlir
    M mlir/test/lib/Dialect/SPIRV/TestModuleCombiner.cpp

  Log Message:
  -----------
  [mlir] Refactor pass crash reproducer generation to be an assembly resource

We currently generate reproducer configurations using a comment placed at
the top of the generated .mlir file. This is kind of hacky given that comments
have no semantic context in the source file and can easily be dropped. This
strategy also wouldn't work if/when we have a bitcode format. This commit
switches to using an external assembly resource, which is verifiable/can
work with a hypothetical bitcode naturally/and removes the awkward processing
from mlir-opt for splicing comments and re-applying command line options. With
the removal of command line munging, this opens up new possibilities for
executing reproducers in memory.

Differential Revision: https://reviews.llvm.org/D126447


Compare: https://github.com/llvm/llvm-project/compare/75095e628124...361acbb36272