[Mlir-commits] [mlir] [mlir][mesh] adding shard-size control (PR #98145)

Wed Aug 7 05:44:47 PDT 2024

llvm-ci wrote:

LLVM Buildbot has detected a new failure on builder `flang-aarch64-libcxx` running on `linaro-flang-aarch64-libcxx` while building `mlir` at step 5 "build-unified-tree".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/89/builds/3580

Here is the relevant piece of the build log for the reference:
```
Step 5 (build-unified-tree) failure: build (failure)
...
179.428 [1549/47/5517] Creating library symlink lib/libMLIRTestTransformDialect.so
179.431 [1549/46/5518] Creating library symlink lib/libMLIROpenMPToLLVMIRTranslation.so
179.464 [1549/45/5519] Linking CXX shared library lib/libMLIRVCIXToLLVMIRTranslation.so.20.0git
179.467 [1549/44/5520] Linking CXX shared library lib/libMLIRX86VectorToLLVMIRTranslation.so.20.0git
179.485 [1549/43/5521] Linking CXX shared library lib/libMLIRTargetLLVM.so.20.0git
179.492 [1544/47/5522] Creating library symlink lib/libMLIRVCIXToLLVMIRTranslation.so
179.492 [1544/46/5523] Creating library symlink lib/libMLIRX86VectorToLLVMIRTranslation.so
179.495 [1540/49/5524] Creating library symlink lib/libMLIRTargetLLVM.so
179.505 [1540/48/5525] Linking CXX shared library lib/libMLIRCAPITransformDialect.so.20.0git
179.509 [1540/47/5526] Linking CXX shared library lib/libMLIRTensorDialect.so.20.0git
FAILED: lib/libMLIRTensorDialect.so.20.0git 
: && /usr/local/bin/c++ -fPIC -stdlib=libc++ -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Wundef -Werror=mismatched-tags -Werror=global-constructors -O3 -DNDEBUG  -stdlib=libc++ -Wl,-z,defs -Wl,-z,nodelete   -Wl,-rpath-link,/home/tcwg-buildbot/worker/flang-aarch64-libcxx/build/./lib  -Wl,--gc-sections -shared -Wl,-soname,libMLIRTensorDialect.so.20.0git -o lib/libMLIRTensorDialect.so.20.0git tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/TensorDialect.cpp.o tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/TensorOps.cpp.o tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ValueBoundsOpInterfaceImpl.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib:/home/tcwg-buildbot/worker/flang-aarch64-libcxx/build/lib:"  lib/libMLIRAffineDialect.so.20.0git  lib/libMLIRParallelCombiningOpInterface.so.20.0git  lib/libMLIRMemRefDialect.so.20.0git  lib/libMLIRArithUtils.so.20.0git  lib/libMLIRComplexDialect.so.20.0git  lib/libMLIRArithAttrToLLVMConversion.so.20.0git  lib/libMLIRArithDialect.so.20.0git  lib/libMLIRCastInterfaces.so.20.0git  lib/libMLIRInferIntRangeCommon.so.20.0git  lib/libMLIRLLVMDialect.so.20.0git  lib/libLLVMBitWriter.so.20.0git  lib/libLLVMAsmParser.so.20.0git  lib/libLLVMBitReader.so.20.0git  lib/libMLIRDialectUtils.so.20.0git  lib/libMLIRShapedOpInterfaces.so.20.0git  lib/libMLIRValueBoundsOpInterface.so.20.0git  lib/libMLIRDestinationStyleOpInterface.so.20.0git  lib/libMLIRAnalysis.so.20.0git  lib/libMLIRInferTypeOpInterface.so.20.0git  lib/libMLIRSideEffectInterfaces.so.20.0git  lib/libMLIRLoopLikeInterface.so.20.0git  lib/libMLIRInferIntRangeInterface.so.20.0git  lib/libMLIRFunctionInterfaces.so.20.0git  lib/libMLIRCallInterfaces.so.20.0git  lib/libMLIRDataLayoutInterfaces.so.20.0git  lib/libMLIRPresburger.so.20.0git  lib/libMLIRViewLikeInterface.so.20.0git  lib/libLLVMCore.so.20.0git  lib/libLLVMBinaryFormat.so.20.0git  lib/libMLIRDialect.so.20.0git  lib/libMLIRControlFlowInterfaces.so.20.0git  lib/libMLIRMemorySlotInterfaces.so.20.0git  lib/libMLIRUBDialect.so.20.0git  lib/libMLIRIR.so.20.0git  lib/libMLIRSupport.so.20.0git  -lpthread  lib/libLLVMSupport.so.20.0git  -Wl,-rpath-link,/home/tcwg-buildbot/worker/flang-aarch64-libcxx/build/lib && :
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::detail::ShardingInterfaceInterfaceTraits::FallbackModel<(anonymous namespace)::EmptyOpShardingInterface>::getShardingOption(mlir::mesh::detail::ShardingInterfaceInterfaceTraits::Concept const*, mlir::Operation*, llvm::ArrayRef<mlir::mesh::MeshSharding>, llvm::ArrayRef<mlir::mesh::MeshSharding>)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir4mesh6detail32ShardingInterfaceInterfaceTraits13FallbackModelIN12_GLOBAL__N_124EmptyOpShardingInterfaceEE17getShardingOptionEPKNS2_7ConceptEPNS_9OperationEN4llvm8ArrayRefINS0_12MeshShardingEEESF_+0x50): undefined reference to `mlir::mesh::detail::defaultGetShardingOption(mlir::Operation*, llvm::ArrayRef<mlir::mesh::MeshSharding>, llvm::ArrayRef<mlir::mesh::MeshSharding>)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::detail::ShardingInterfaceInterfaceTraits::FallbackModel<(anonymous namespace)::EmptyOpShardingInterface>::getShardingAnnotations(mlir::mesh::detail::ShardingInterfaceInterfaceTraits::Concept const*, mlir::Operation*, mlir::mesh::ShardingOption const&)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir4mesh6detail32ShardingInterfaceInterfaceTraits13FallbackModelIN12_GLOBAL__N_124EmptyOpShardingInterfaceEE22getShardingAnnotationsEPKNS2_7ConceptEPNS_9OperationERKNS0_14ShardingOptionE+0x44): undefined reference to `mlir::mesh::detail::defaultGetShardingAnnotations(mlir::Operation*, mlir::mesh::ShardingOption const&)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::detail::ShardingInterfaceInterfaceTraits::FallbackModel<(anonymous namespace)::EmptyOpShardingInterface>::addShardingAnnotations(mlir::mesh::detail::ShardingInterfaceInterfaceTraits::Concept const*, mlir::Operation*, mlir::OpBuilder&, mlir::mesh::ShardingOption const&)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir4mesh6detail32ShardingInterfaceInterfaceTraits13FallbackModelIN12_GLOBAL__N_124EmptyOpShardingInterfaceEE22addShardingAnnotationsEPKNS2_7ConceptEPNS_9OperationERNS_9OpBuilderERKNS0_14ShardingOptionE+0x48): undefined reference to `mlir::mesh::detail::defaultAddShardingAnnotations(mlir::Operation*, mlir::OpBuilder&, mlir::mesh::ShardingOption const&)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::detail::ShardingInterfaceInterfaceTraits::FallbackModel<(anonymous namespace)::EmptyOpShardingInterface>::spmdize(mlir::mesh::detail::ShardingInterfaceInterfaceTraits::Concept const*, mlir::Operation*, llvm::ArrayRef<mlir::Value>, llvm::ArrayRef<mlir::mesh::MeshSharding>, llvm::ArrayRef<mlir::mesh::MeshSharding>, mlir::IRMapping&, mlir::SymbolTableCollection&, mlir::OpBuilder&)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir4mesh6detail32ShardingInterfaceInterfaceTraits13FallbackModelIN12_GLOBAL__N_124EmptyOpShardingInterfaceEE7spmdizeEPKNS2_7ConceptEPNS_9OperationEN4llvm8ArrayRefINS_5ValueEEENSD_INS0_12MeshShardingEEESH_RNS_9IRMappingERNS_21SymbolTableCollectionERNS_9OpBuilderE+0x7c): undefined reference to `mlir::mesh::shardType(mlir::Type, mlir::mesh::MeshOp, mlir::mesh::MeshSharding)'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4mlir4mesh6detail32ShardingInterfaceInterfaceTraits13FallbackModelIN12_GLOBAL__N_124EmptyOpShardingInterfaceEE7spmdizeEPKNS2_7ConceptEPNS_9OperationEN4llvm8ArrayRefINS_5ValueEEENSD_INS0_12MeshShardingEEESH_RNS_9IRMappingERNS_21SymbolTableCollectionERNS_9OpBuilderE+0x388): undefined reference to `mlir::mesh::ShardShapeOp::getODSResultIndexAndLength(unsigned int)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::ShardingOp mlir::OpBuilder::create<mlir::mesh::ShardingOp, mlir::mesh::MeshSharding const&>(mlir::Location, mlir::mesh::MeshSharding const&)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh10ShardingOpEJRKNS2_12MeshShardingEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh10ShardingOpEJRKNS2_12MeshShardingEEEET_NS_8LocationEDpOT0_]+0x30): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardingOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh10ShardingOpEJRKNS2_12MeshShardingEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh10ShardingOpEJRKNS2_12MeshShardingEEEET_NS_8LocationEDpOT0_]+0x34): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardingOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh10ShardingOpEJRKNS2_12MeshShardingEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh10ShardingOpEJRKNS2_12MeshShardingEEEET_NS_8LocationEDpOT0_]+0x70): undefined reference to `mlir::mesh::ShardingOp::build(mlir::OpBuilder&, mlir::OperationState&, mlir::mesh::MeshSharding)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::ProcessLinearIndexOp mlir::OpBuilder::create<mlir::mesh::ProcessLinearIndexOp, llvm::StringRef>(mlir::Location, llvm::StringRef&&)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh20ProcessLinearIndexOpEJN4llvm9StringRefEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh20ProcessLinearIndexOpEJN4llvm9StringRefEEEET_NS_8LocationEDpOT0_]+0x30): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ProcessLinearIndexOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh20ProcessLinearIndexOpEJN4llvm9StringRefEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh20ProcessLinearIndexOpEJN4llvm9StringRefEEEET_NS_8LocationEDpOT0_]+0x34): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ProcessLinearIndexOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh20ProcessLinearIndexOpEJN4llvm9StringRefEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh20ProcessLinearIndexOpEJN4llvm9StringRefEEEET_NS_8LocationEDpOT0_]+0x60): undefined reference to `mlir::mesh::ProcessLinearIndexOp::build(mlir::OpBuilder&, mlir::OperationState&, llvm::StringRef)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `mlir::mesh::ShardShapeOp mlir::OpBuilder::create<mlir::mesh::ShardShapeOp, llvm::ArrayRef<long>, mlir::OpResult, mlir::Value&>(mlir::Location, llvm::ArrayRef<long>&&, mlir::OpResult&&, mlir::Value&)':
ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh12ShardShapeOpEJN4llvm8ArrayRefIlEENS_8OpResultERNS_5ValueEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh12ShardShapeOpEJN4llvm8ArrayRefIlEENS_8OpResultERNS_5ValueEEEET_NS_8LocationEDpOT0_]+0x3c): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardShapeOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh12ShardShapeOpEJN4llvm8ArrayRefIlEENS_8OpResultERNS_5ValueEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh12ShardShapeOpEJN4llvm8ArrayRefIlEENS_8OpResultERNS_5ValueEEEET_NS_8LocationEDpOT0_]+0x40): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardShapeOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4mlir9OpBuilder6createINS_4mesh12ShardShapeOpEJN4llvm8ArrayRefIlEENS_8OpResultERNS_5ValueEEEET_NS_8LocationEDpOT0_[_ZN4mlir9OpBuilder6createINS_4mesh12ShardShapeOpEJN4llvm8ArrayRefIlEENS_8OpResultERNS_5ValueEEEET_NS_8LocationEDpOT0_]+0x74): undefined reference to `mlir::mesh::ShardShapeOp::build(mlir::OpBuilder&, mlir::OperationState&, llvm::ArrayRef<long>, mlir::Value, mlir::Value)'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `llvm::DefaultDoCastIfPossible<mlir::mesh::MeshOp, mlir::Operation*, llvm::CastInfo<mlir::mesh::MeshOp, mlir::Operation*, void> >::doCastIfPossible(mlir::Operation*)':
ShardingInterfaceImpl.cpp:(.text._ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh6MeshOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_[_ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh6MeshOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_]+0x24): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::MeshOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh6MeshOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_[_ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh6MeshOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_]+0x28): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::MeshOp, void>::id'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `llvm::DefaultDoCastIfPossible<mlir::mesh::ShardingOp, mlir::Operation*, llvm::CastInfo<mlir::mesh::ShardingOp, mlir::Operation*, void> >::doCastIfPossible(mlir::Operation*)':
ShardingInterfaceImpl.cpp:(.text._ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh10ShardingOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_[_ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh10ShardingOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_]+0x24): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardingOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh10ShardingOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_[_ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh10ShardingOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_]+0x28): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardingOp, void>::id'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `llvm::DefaultDoCastIfPossible<mlir::mesh::ProcessLinearIndexOp, mlir::Operation*, llvm::CastInfo<mlir::mesh::ProcessLinearIndexOp, mlir::Operation*, void> >::doCastIfPossible(mlir::Operation*)':
ShardingInterfaceImpl.cpp:(.text._ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh20ProcessLinearIndexOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_[_ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh20ProcessLinearIndexOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_]+0x24): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ProcessLinearIndexOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh20ProcessLinearIndexOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_[_ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh20ProcessLinearIndexOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_]+0x28): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ProcessLinearIndexOp, void>::id'
/usr/bin/ld: tools/mlir/lib/Dialect/Tensor/IR/CMakeFiles/obj.MLIRTensorDialect.dir/ShardingInterfaceImpl.cpp.o: in function `llvm::DefaultDoCastIfPossible<mlir::mesh::ShardShapeOp, mlir::Operation*, llvm::CastInfo<mlir::mesh::ShardShapeOp, mlir::Operation*, void> >::doCastIfPossible(mlir::Operation*)':
ShardingInterfaceImpl.cpp:(.text._ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh12ShardShapeOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_[_ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh12ShardShapeOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_]+0x24): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardShapeOp, void>::id'
/usr/bin/ld: ShardingInterfaceImpl.cpp:(.text._ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh12ShardShapeOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_[_ZN4llvm23DefaultDoCastIfPossibleIN4mlir4mesh12ShardShapeOpEPNS1_9OperationENS_8CastInfoIS3_S5_vEEE16doCastIfPossibleES5_]+0x28): undefined reference to `mlir::detail::TypeIDResolver<mlir::mesh::ShardShapeOp, void>::id'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
179.523 [1540/46/5527] Linking CXX shared library lib/libMLIROpenACCTransforms.so.20.0git
179.533 [1540/45/5528] Linking CXX shared library lib/libMLIROpenACCToLLVMIRTranslation.so.20.0git
179.540 [1540/44/5529] Linking CXX shared library lib/libMLIRGPUToLLVMIRTranslation.so.20.0git

```

https://github.com/llvm/llvm-project/pull/98145