[Mlir-commits] [mlir] [mlir][mesh] adding shard-size control (PR #98145)

Wed Aug 7 06:38:05 PDT 2024

llvm-ci wrote:

LLVM Buildbot has detected a new failure on builder `flang-aarch64-latest-gcc` running on `linaro-flang-aarch64-latest-gcc` while building `mlir` at step 5 "build-unified-tree".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/130/builds/1817

Here is the relevant piece of the build log for the reference:
```
Step 5 (build-unified-tree) failure: build (failure)
...
1422.171 [2898/17/4214] Creating library symlink lib/libMLIRNVGPUDialect.so
1423.046 [2898/16/4215] Building CXX object tools/mlir/lib/Dialect/XeGPU/IR/CMakeFiles/obj.MLIRXeGPUDialect.dir/XeGPUDialect.cpp.o
1423.560 [2898/15/4216] Building CXX object tools/mlir/lib/Dialect/X86Vector/Transforms/CMakeFiles/obj.MLIRX86VectorTransforms.dir/AVXTranspose.cpp.o
1424.535 [2898/14/4217] Building CXX object tools/mlir/lib/Target/Cpp/CMakeFiles/obj.MLIRTargetCpp.dir/TranslateToCpp.cpp.o
1424.547 [2896/15/4218] Creating library symlink lib/libMLIRAMDGPUUtils.so
1424.557 [2894/16/4219] Creating library symlink lib/libMLIROpenACCDialect.so
1424.791 [2894/15/4220] Linking CXX shared library lib/libMLIRX86VectorDialect.so.20.0git
1424.813 [2893/15/4221] Creating library symlink lib/libMLIRX86VectorDialect.so
1425.424 [2893/14/4222] Building CXX object tools/mlir/lib/Dialect/XeGPU/IR/CMakeFiles/obj.MLIRXeGPUDialect.dir/XeGPUOps.cpp.o
1425.578 [2893/13/4223] Linking CXX shared library lib/libMLIRTensorDialect.so.20.0git
FAILED: lib/libMLIRTensorDialect.so.20.0git 
: && /usr/local/bin/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -fno-lifetime-dse -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-nonnull -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Wundef -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/home/tcwg-buildbot/worker/flang-aarch64-latest-gcc/build/./lib  -Wl,--gc-sections -shared -Wl,-soname,libMLIRTensorDialect.so.20.0git -o lib/libMLIRTensorDialect.so.20.0git tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/TensorDialect.cpp.o tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/TensorOps.cpp.o tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ValueBoundsOpInterfaceImpl.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib:/home/tcwg-buildbot/worker/flang-aarch64-latest-gcc/build/lib:"  lib/libMLIRAffineDialect.so.20.0git  lib/libMLIRParallelCombiningOpInterface.so.20.0git  lib/libMLIRMemRefDialect.so.20.0git  lib/libMLIRArithUtils.so.20.0git  lib/libMLIRComplexDialect.so.20.0git  lib/libMLIRArithAttrToLLVMConversion.so.20.0git  lib/libMLIRArithDialect.so.20.0git  lib/libMLIRCastInterfaces.so.20.0git  lib/libMLIRInferIntRangeCommon.so.20.0git  lib/libMLIRLLVMDialect.so.20.0git  lib/libLLVMBitWriter.so.20.0git  lib/libLLVMAsmParser.so.20.0git  lib/libLLVMBitReader.so.20.0git  lib/libMLIRDialectUtils.so.20.0git  lib/libMLIRShapedOpInterfaces.so.20.0git  lib/libMLIRValueBoundsOpInterface.so.20.0git  lib/libMLIRDestinationStyleOpInterface.so.20.0git  lib/libMLIRAnalysis.so.20.0git  lib/libMLIRInferTypeOpInterface.so.20.0git  lib/libMLIRSideEffectInterfaces.so.20.0git  lib/libMLIRLoopLikeInterface.so.20.0git  lib/libMLIRInferIntRangeInterface.so.20.0git  lib/libMLIRFunctionInterfaces.so.20.0git  lib/libMLIRCallInterfaces.so.20.0git  lib/libMLIRDataLayoutInterfaces.so.20.0git  lib/libMLIRPresburger.so.20.0git  lib/libMLIRViewLikeInterface.so.20.0git  lib/libLLVMCore.so.20.0git  lib/libLLVMBinaryFormat.so.20.0git  lib/libMLIRDialect.so.20.0git  lib/libMLIRControlFlowInterfaces.so.20.0git  lib/libMLIRMemorySlotInterfaces.so.20.0git  lib/libMLIRUBDialect.so.20.0git  lib/libMLIRIR.so.20.0git  lib/libMLIRSupport.so.20.0git  -lpthread  lib/libLLVMSupport.so.20.0git  -Wl,-rpath-link,/home/tcwg-buildbot/worker/flang-aarch64-latest-gcc/build/lib && :
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::detail::ShardingInterfaceInterfaceTraits::FallbackModel<(anonymous namespace)::EmptyOpShardingInterface>::getShardingAnnotations(mlir::mesh::detail::ShardingInterfaceInterfaceTraits::Concept const*, mlir::Operation*, mlir::mesh::ShardingOption const&)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir4mesh6detail32ShardingInterfaceInterfaceTraits13FallbackModelIN12_GLOBAL__N_124EmptyOpShardingInterfaceEE22getShardingAnnotationsEPKNS2_7ConceptEPNS_9OperationERKNS0_14ShardingOptionE+0x54): undefined reference to `mlir::mesh::detail::defaultGetShardingAnnotations(mlir::Operation*, mlir::mesh::ShardingOption const&)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::detail::ShardingInterfaceInterfaceTraits::FallbackModel<(anonymous namespace)::EmptyOpShardingInterface>::addShardingAnnotations(mlir::mesh::detail::ShardingInterfaceInterfaceTraits::Concept const*, mlir::Operation*, mlir::OpBuilder&, mlir::mesh::ShardingOption const&)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir4mesh6detail32ShardingInterfaceInterfaceTraits13FallbackModelIN12_GLOBAL__N_124EmptyOpShardingInterfaceEE22addShardingAnnotationsEPKNS2_7ConceptEPNS_9OperationERNS_9OpBuilderERKNS0_14ShardingOptionE+0x70): undefined reference to `mlir::mesh::detail::defaultAddShardingAnnotations(mlir::Operation*, mlir::OpBuilder&, mlir::mesh::ShardingOption const&)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::detail::ShardingInterfaceInterfaceTraits::FallbackModel<(anonymous namespace)::EmptyOpShardingInterface>::getShardingOption(mlir::mesh::detail::ShardingInterfaceInterfaceTraits::Concept const*, mlir::Operation*, llvm::ArrayRef<mlir::mesh::MeshSharding>, llvm::ArrayRef<mlir::mesh::MeshSharding>)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir4mesh6detail32ShardingInterfaceInterfaceTraits13FallbackModelIN12_GLOBAL__N_124EmptyOpShardingInterfaceEE17getShardingOptionEPKNS2_7ConceptEPNS_9OperationEN4llvm8ArrayRefINS0_12MeshShardingEEESF_+0x60): undefined reference to `mlir::mesh::detail::defaultGetShardingOption(mlir::Operation*, llvm::ArrayRef<mlir::mesh::MeshSharding>, llvm::ArrayRef<mlir::mesh::MeshSharding>)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `(anonymous namespace)::EmptyOpShardingInterface::spmdize(mlir::Operation*, llvm::ArrayRef<mlir::Value>, llvm::ArrayRef<mlir::mesh::MeshSharding>, llvm::ArrayRef<mlir::mesh::MeshSharding>, mlir::IRMapping&, mlir::SymbolTableCollection&, mlir::OpBuilder&) const [clone .isra.0]':
ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x94): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::MeshOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x98): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::MeshOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x184): undefined reference to `mlir::mesh::shardType(mlir::Type, mlir::mesh::MeshOp, mlir::mesh::MeshSharding)'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x534): undefined reference to `mlir::mesh::ShardShapeOp::getODSResultIndexAndLength(unsigned int)'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x698): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardingOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x69c): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardingOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x788): undefined reference to `mlir::mesh::ShardingOp::build(mlir::OpBuilder&, mlir::OperationState&, mlir::mesh::MeshSharding)'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x82c): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardingOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x830): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardingOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x86c): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ProcessLinearIndexOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x870): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ProcessLinearIndexOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x8b4): undefined reference to `mlir::mesh::ProcessLinearIndexOp::build(mlir::OpBuilder&, mlir::OperationState&, llvm::StringRef)'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x94c): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardShapeOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x950): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardShapeOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZNK12_GLOBAL__N_124EmptyOpShardingInterface7spmdizeEPN4mlir9OperationEN4llvm8ArrayRefINS1_5ValueEEENS5_INS1_4mesh12MeshShardingEEESA_RNS1_9IRMappingERNS1_21SymbolTableCollectionERNS1_9OpBuilderE.isra.0+0x998): undefined reference to `mlir::mesh::ShardShapeOp::build(mlir::OpBuilder&, mlir::OperationState&, llvm::ArrayRef<long>, mlir::Value, mlir::Value)'
collect2: error: ld returned 1 exit status
1427.453 [2893/12/4224] Building CXX object tools/mlir/lib/Target/SPIRV/Serialization/CMakeFiles/obj.MLIRSPIRVSerialization.dir/Serializer.cpp.o
1428.787 [2893/11/4225] Building CXX object tools/mlir/lib/Target/LLVMIR/CMakeFiles/obj.MLIRTargetLLVMIRExport.dir/DebugTranslation.cpp.o
1428.903 [2893/10/4226] Building CXX object tools/mlir/lib/Dialect/Vector/Transforms/CMakeFiles/obj.MLIRVectorTransforms.dir/VectorEmulateMaskedLoadStore.cpp.o
1429.051 [2893/9/4227] Building CXX object tools/mlir/lib/Target/LLVMIR/CMakeFiles/obj.MLIRTargetLLVMIRExport.dir/LoopAnnotationTranslation.cpp.o
1430.746 [2893/8/4228] Building CXX object tools/mlir/lib/Dialect/SPIRV/IR/CMakeFiles/obj.MLIRSPIRVDialect.dir/SPIRVOpDefinition.cpp.o
1434.025 [2893/7/4229] Building CXX object tools/mlir/lib/Dialect/Vector/TransformOps/CMakeFiles/obj.MLIRVectorTransformOps.dir/VectorTransformOps.cpp.o
1437.915 [2893/6/4230] Building CXX object tools/mlir/lib/Dialect/Vector/Transforms/CMakeFiles/obj.MLIRVectorTransforms.dir/VectorEmulateNarrowType.cpp.o
In file included from ../llvm-project/mlir/include/mlir/Rewrite/FrozenRewritePatternSet.h:12,
                 from ../llvm-project/mlir/include/mlir/Transforms/DialectConversion.h:17,
                 from ../llvm-project/mlir/include/mlir/Dialect/Arith/Transforms/NarrowTypeEmulationConverter.h:13,
                 from ../llvm-project/mlir/lib/Dialect/Vector/Transforms/VectorEmulateNarrowType.cpp:12:
../llvm-project/mlir/include/mlir/Transforms/DialectConversion.h: In instantiation of ‘class mlir::OpConversionPattern<mlir::vector::StoreOp>’:
../llvm-project/mlir/lib/Dialect/Vector/Transforms/VectorEmulateNarrowType.cpp:111:35:   required from here
../llvm-project/mlir/include/mlir/IR/PatternMatch.h:255:16: warning: ‘virtual void mlir::RewritePattern::rewrite(mlir::Operation*, mlir::PatternRewriter&) const’ was hidden [-Woverloaded-virtual=]

```

https://github.com/llvm/llvm-project/pull/98145