[Mlir-commits] [mlir] [mlir] Declare promised interfaces for all dialects (PR #78368)

[Markus Böck]

zero9178 wrote:

I am seeing some build failures with this PR when testing with `ninja -t missingdeps`. In addition to some preexisting issues in MLIR/LLVM the following missing dependencies are added:
```
Missing dep: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/TensorDialect.cpp.obj uses tools/mlir/include/mlir/Dialect/Tensor/TransformOps/TensorTransformOps.h.inc (generated by CUSTOM_COMMAND)
Missing dep: tools/mlir/lib/Dialect/Tosa/CMakeFiles/obj.MLIRTosaDialect.dir/IR/TosaOps.cpp.obj uses tools/mlir/include/mlir/Dialect/Mesh/Interfaces/ShardingInterface.h.inc (generated by CUSTOM_COMMAND)
Missing dep: tools/mlir/lib/Dialect/Tosa/CMakeFiles/obj.MLIRTosaDialect.dir/IR/TosaOps.cpp.obj uses tools/mlir/include/mlir/Dialect/Mesh/IR/MeshOpsAttributes.h.inc (generated by CUSTOM_COMMAND)
Missing dep: tools/mlir/lib/Dialect/Tosa/CMakeFiles/obj.MLIRTosaDialect.dir/IR/TosaOps.cpp.obj uses tools/mlir/include/mlir/Dialect/Mesh/IR/MeshOpsEnums.h.inc (generated by CUSTOM_COMMAND)
```
This is due to cpp files such as `TensorDialect.cpp` now having a dependency on an interface and its `mlir-tblgen` generated header but this dependency not being present in CMake. This can cause build flakiness if the build system where to schedule the compilation of `TensorDialect.cpp` first or if one were to build targets that requires building `TensorDialect.cpp` but not the interface it depends on, causing the interface generation to never be scheduled. 
To fix this the cmake libraries that contain a cpp file should have an explicit dependency on the interface generation target. 
For `TosaOps.cpp` this would e.g. be done by adding `MLIRTensorTransformOpsIncGen` to the `DEPENDS` list in `mlir/lib/Dialect/Tensor/IR/CMakeLists.txt`

To reproduce: Make sure to use `-G Ninja` as generator, run a full build of MLIR (I like to do `ninja check-mlir`) and then run `ninja -t missingdeps`. There will be other pre-existing issues besides the ones cause by this PR.

To make sure its not a false positive you can also manually create build failures. To do so:
1) Run `ninja clean`. This should also delete the `*.h.inc` and `*.cpp.inc` files from the interfaces.
2) Run `rm .ninja_deps .ninja_log`. Ninja build is stateful and logs header dependencies. IIRC for better scheduling. If you regularly used the generation directory it may have already noted the build dependency and would then schedule building the interface header.
3) Build one of the targets shown previously. E.g. `ninja MLIRTensorDialect`. On my machine this creates a build error:
```
ccache C:\clang\Default\bin\clang-cl.exe  /nologo -TP -DGTEST_HAS_RTTI=0 -DMLIR_CUDA_CONVERSIONS_ENABLED=1 -DMLIR_ROCM_CONVERSIONS_ENABLED=1 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRE
CATE -D_CRT_SECURE_NO_WARNINGS -D_GLIBCXX_ASSERTIONS -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -IG:/CLion/llvm-proje
ct/llvm/cmake-build-release-clang-visual-studio/tools/mlir/lib/Dialect/Tensor/IR -IG:/CLion/llvm-project/mlir/lib/Dialect/Tensor/IR -IG:/CLion/llvm-project/llvm/cmake-build-release-clang-visual-studio/include -IG:/CLion/llvm-pro
ject/llvm/include -IG:/CLion/llvm-project/mlir/include -IG:/CLion/llvm-project/llvm/cmake-build-release-clang-visual-studio/tools/mlir/include -IC:/Libraries/MSVC-14x/zlib/include -D_SILENCE_NONFLOATING_COMPLEX_DEPRECATION_WARNI
NG /Zc:inline /Zc:__cplusplus /Zi -gcodeview-ghash /Oi /Brepro /bigobj /permissive- /W4  -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno
-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported /Gw -Werror=mismatched-tags -Werror=global-constructors /O2 /Ob2  -std:c++17 -M
D  /EHs-c- /GR- -UNDEBUG /showIncludes /Fotools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/TensorDialect.cpp.obj /Fdtools\mlir\lib\Dialect\Tensor\IR\CMakeFiles\obj.MLIRTensorDialect.dir\ -c -- G:/CLion/llvm-project/mlir/lib/Dialect/Tensor/IR/TensorDialect.cpp
In file included from G:/CLion/llvm-project/mlir/lib/Dialect/Tensor/IR/TensorDialect.cpp:14:
G:/CLion/llvm-project/mlir/include\mlir/Dialect/Tensor/TransformOps/TensorTransformOps.h(28,10): fatal error: 'mlir/Dialect/Tensor/TransformOps/TensorTransformOps.h.inc' file not found
   28 | #include "mlir/Dialect/Tensor/TransformOps/TensorTransformOps.h.inc"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

I would also recommend the use of `ccache` so that the build after running `ninja clean` is done in a few seconds

https://github.com/llvm/llvm-project/pull/78368