[Mlir-commits] [mlir] Add a tutorial on mlir-opt (PR #96105)

[Jeremy Kun]

================
@@ -0,0 +1,432 @@
+# Using `mlir-opt`
+
+`mlir-opt` is the command-line entry point for running passes and lowerings on MLIR code.
+This tutorial will explain how to use `mlir-opt` to run passes, and explain
+some details about MLIR's built-in dialects along the way.
+
+Prerequisites:
+
+- [Building MLIR from source](/getting_started/)
+
+[TOC]
+
+## Overview
+
+We start with a brief summary of context that helps to frame
+the uses of `mlir-opt` detailed in this article.
+For a deeper dive on motivation and design,
+see [the MLIR paper](https://arxiv.org/abs/2002.11054).
+
+Two of the central concepts in MLIR are *dialects* and *lowerings*.
+In traditional compilers, there is typically one "dialect,"
+called an *intermediate representation*, or IR,
+that is the textual or data-structural description of a program
+within the scope of the compiler's execution.
+For example, in GCC the IR is called GIMPLE,
+and in LLVM it's called LLVM-IR.
+Compilers typically convert an input program to their IR,
+run optimization passes,
+and then convert the optimized IR to machine code.
+
+MLIR's philosophy is to split the job into smaller steps.
+First, MLIR allows one to define many IRs called *dialects*,
+some considered "high level" and some "low level,"
+but each with a set of types, operations, metadata,
+and semantics that defines what the operations do.
+Different dialects may coexist in the same program.
+Then, one writes a set of *lowering passes*
+that incrementally converts different parts of the program
+from higher level dialects to lower and lower dialects
+until you get to machine code
+(or, in many cases, LLVM, which finishes the job).
+Along the way,
+*optimizing passes* are run to make the code more efficient.
+The main point here is that the high level dialects exist
+*so that* they make it easy to write these important optimizing passes.
+
+A central motivation for building MLIR
+was to build the `affine` dialect,
+which is designed to enable [polyhedral optimizations](https://polyhedral.info/)
+for loop transformations.
+Compiler engineers had previously implemented polyhedral optimizations
+in LLVM and GCC (without an `affine` dialect),
+and it was difficult because they had to reconstruct well-structured loop nests
+from a much more complicated set of low-level operations.
+Having a higher level `affine` dialect preserves the loop nest structure
+at an abstraction layer that makes it easier to write optimizations,
+and then discards it during lowering passes.
+
+The `mlir-opt` tool can run both
+optimization passes and lowerings,
+though the final code generation
+is performed by a different tool called `mlir-translate`.
+In particular, `mlir-opt` consumes MLIR as input and produce MLIR as output,
+while `mlir-translate` consumes MLIR as input
+and produces non-MLIR program representations as output.
+
+## Two example programs
+
+Here are two MLIR programs.
+The first defines a function that counts the leading zeroes of a 32-bit integer (`i32`)
+using the [`math` dialect's](/docs/Dialects/MathOps/) `ctlz` operation.
+
+```mlir
+// mlir/test/Examples/mlir-opt/ctlz.mlir
+func.func @main(%arg0: i32) -> i32 {
+  %0 = math.ctlz %arg0 : i32
+  func.return %0 : i32
+}
+```
+
+This shows the basic structure of an MLIR operation
+([see here](https://mlir.llvm.org/docs/LangRef/#operations) for a more complete spec).
+Variable names are prefixed with `%`,
+functions by `@`,
+and each variable/value in a program has a type,
+often expressed after a colon.
+In this case all the types are `i32`,
+except for the function type which is `(i32) -> i32`
+(not specified explicitly above, but you'll see it in the `func.call` later).
+
+Each statement is anchored around an expression like `math.ctlz`
+which specifies the dialect [`math`](https://mlir.llvm.org/docs/Dialects/MathOps/) via a namespace,
+and the operation [`ctlz`](https://mlir.llvm.org/docs/Dialects/MathOps/#mathctlz-mathcountleadingzerosop) after the `.`.
+The rest of the syntax of the operation
+is determined by a parser defined by the dialect,
+and so many operations will have different syntaxes.
+In the case of `math.ctlz`,
+the sole argument is an integer whose leading zeros are to be counted,
+and the trailing ` : i32` denotes the output type storing the count.
----------------
j2kun wrote:

I should have added this explanation to the PR description (I will copy this comment there for visibility).

The goal of this document is to provide the user the first thing they can do after building MLIR. As such, it is designed to get them started on their MLIR journey and hand-hold them through baby steps. It should be self-contained and have no strict dependencies on other documentation. IMO it's also acceptable that this tutorial differs slightly from the way an experienced person would do the same task. [A tutorial for beginners is not the same thing as a manual for best practice.](https://docs.divio.com/documentation-system/tutorials/#how-to-write-good-tutorials)

I want to demonstrate how to use `mlir-opt` to run both a lowering and a within-dialect optimization pass, and to explain the terminology required to demonstrate that. For example, showing a pass on the affine dialect requires me to explain, at least in brief, what the affine dialect is and why it exists. Without that sort of context, this would be less of a tutorial and more of a reference document (which can be good, but is not what I'm trying to do here). Toy Chapter 2 does better, but that still attempts to be exhaustive and spends a lot of time connecting the exhaustive description to MLIR's design rationale, whereas here I'm trying to explain the minimum possible to be able to read the programs in the examples. Also, I want this document to be meaningfully different from the help text of `mlir-opt`, in that it contains context not suitable for that help text, and also focuses on a very small subset of the available capability of `mlir-opt`. 

Generally speaking, I think it's much better to spend a paragraph or two to introduce a topic rather than ask the reader to go read a much longer reference spec before they can proceed. E.g., the MLIR language reference, which is written in a very comprehensive and generic fashion, introduces a few dozen concepts in its first four paragraphs, before giving a 60-line example program for a relatively complicated task that also depends in some sense on knowledge of TensorFlow.

It's also good to have tutorials with overlapping subsets of information.

https://github.com/llvm/llvm-project/pull/96105