[Mlir-commits] [mlir] [mlir-translate] Add structured translation entry point (PR #222623)
Anutosh Bhat
llvmlistbot at llvm.org
Thu Sep 10 05:11:06 PDT 2026
https://github.com/anutosh491 created https://github.com/llvm/llvm-project/pull/222623
Please read: [RFC: Embeddable LLVM tool drivers for long-lived hosts](https://discourse.llvm.org/t/rfc-embeddable-llvm-tool-drivers-for-long-lived-hosts/91754) and the [MLIR section](https://discourse.llvm.org/t/rfc-embeddable-llvm-tool-drivers-for-long-lived-hosts/91754#p-367966-mlir-13).
`mlir-opt` already separates its standalone command-line frontend from its reusable implementation:
```text
command-line parsing
↓
MlirOptMainConfig
↓
MlirOptMain(input buffer, output stream, registry, config)
```
`mlir-translate` currently combines those responsibilities inside `mlirTranslateMain(int argc, char **argv, ...)`: it registers and parses its command-line options, opens the input and output files, and then performs the translations.
This patch applies the existing mlir-opt structure to mlir-translate. It extracts the MLIR translation executor behind a structured overload accepting an input buffer, output stream, selected Translation objects and an MlirTranslateMainConfig. The existing command-line entry point continues to parse the same options and delegates to the new overload, so its command-line behaviour remains unchanged.
Separating parsing from execution is useful for any application embedding MLIR. Such an application already has its input, output and selected translation, and should not be required to install another command-line frontend into the process merely to call MLIRTranslateLib.
[WasmBolt](https://github.com/anutosh491/WasmBolt)(try [here](https://anutosh21.github.io/WasmBolt/)) is one concrete consumer. It runs `mlir-opt`, `opt`, `llc`, Clang and LLD inside one long-lived browser-hosted LLVM process. Calling the complete `mlir-opt` and `mlir-translate` command-line frontends in that process causes common options such as `-o` to be registered twice in LLVM's process-global command-line registry.
```
mlir-translate: CommandLine Error: Option 'o' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options
```
The structured entry point allows WasmBolt to invoke the actual MLIR translation operation without registering another command-line frontend. This is needed to continue naturally from the MLIR LLVM dialect to LLVM IR inside the same compiler session.
Add a simple test where the test register its own `-o` option first, then runs two translations through the new API and checks the output. This confirms that the translation path works without touching the command-line parser. **Can Improve it a bit if it looks too informal.**
>From c37b2638cc4146b9e36cf7a1634847cee0d83f18 Mon Sep 17 00:00:00 2001
From: anutosh491 <andersonbhat491 at gmail.com>
Date: Thu, 10 Sep 2026 17:28:06 +0530
Subject: [PATCH] [mlir-translate] Add structured translation entry point
---
.../Tools/mlir-translate/MlirTranslateMain.h | 70 +++++++
.../mlir-translate/MlirTranslateMain.cpp | 181 +++++++++++-------
mlir/unittests/CMakeLists.txt | 1 +
mlir/unittests/Tools/CMakeLists.txt | 1 +
.../Tools/mlir-translate/CMakeLists.txt | 8 +
.../mlir-translate/MlirTranslateMainTest.cpp | 46 +++++
6 files changed, 235 insertions(+), 72 deletions(-)
create mode 100644 mlir/unittests/Tools/CMakeLists.txt
create mode 100644 mlir/unittests/Tools/mlir-translate/CMakeLists.txt
create mode 100644 mlir/unittests/Tools/mlir-translate/MlirTranslateMainTest.cpp
diff --git a/mlir/include/mlir/Tools/mlir-translate/MlirTranslateMain.h b/mlir/include/mlir/Tools/mlir-translate/MlirTranslateMain.h
index c9bc58bacdace..13b2bc1618702 100644
--- a/mlir/include/mlir/Tools/mlir-translate/MlirTranslateMain.h
+++ b/mlir/include/mlir/Tools/mlir-translate/MlirTranslateMain.h
@@ -13,10 +13,80 @@
#ifndef MLIR_TOOLS_MLIRTRANSLATE_MLIRTRANSLATEMAIN_H
#define MLIR_TOOLS_MLIRTRANSLATE_MLIRTRANSLATEMAIN_H
+#include "mlir/IR/Diagnostics.h"
#include "mlir/Support/LLVM.h"
+#include "mlir/Support/ToolUtilities.h"
#include "llvm/ADT/StringRef.h"
+#include <memory>
+#include <string>
+#include <utility>
+
+namespace llvm {
+class MemoryBuffer;
+class raw_ostream;
+} // namespace llvm
+
namespace mlir {
+class Translation;
+
+/// Configuration options for the mlir-translate driver.
+class MlirTranslateMainConfig {
+public:
+ MlirTranslateMainConfig &allowUnregisteredDialects(bool allow) {
+ allowUnregisteredDialectsFlag = allow;
+ return *this;
+ }
+ bool shouldAllowUnregisteredDialects() const {
+ return allowUnregisteredDialectsFlag;
+ }
+
+ MlirTranslateMainConfig &errorDiagnosticsOnly(bool errorOnly) {
+ errorDiagnosticsOnlyFlag = errorOnly;
+ return *this;
+ }
+ bool shouldEmitErrorDiagnosticsOnly() const {
+ return errorDiagnosticsOnlyFlag;
+ }
+
+ MlirTranslateMainConfig &
+ splitInputFile(std::string splitMarker = kDefaultSplitMarker) {
+ inputSplitMarker = std::move(splitMarker);
+ return *this;
+ }
+ StringRef getInputSplitMarker() const { return inputSplitMarker; }
+
+ MlirTranslateMainConfig &outputSplitMarker(std::string splitMarker) {
+ outputSplitMarkerFlag = std::move(splitMarker);
+ return *this;
+ }
+ StringRef getOutputSplitMarker() const { return outputSplitMarkerFlag; }
+
+ MlirTranslateMainConfig &
+ verifyDiagnostics(SourceMgrDiagnosticVerifierHandler::Level level) {
+ verifyDiagnosticsLevel = level;
+ return *this;
+ }
+ SourceMgrDiagnosticVerifierHandler::Level getVerifyDiagnosticsLevel() const {
+ return verifyDiagnosticsLevel;
+ }
+
+private:
+ bool allowUnregisteredDialectsFlag = false;
+ bool errorDiagnosticsOnlyFlag = false;
+ std::string inputSplitMarker;
+ std::string outputSplitMarkerFlag;
+ SourceMgrDiagnosticVerifierHandler::Level verifyDiagnosticsLevel =
+ SourceMgrDiagnosticVerifierHandler::Level::None;
+};
+
+/// Apply the requested translations to an input buffer and write the result to
+/// the output stream.
+LogicalResult mlirTranslateMain(std::unique_ptr<llvm::MemoryBuffer> input,
+ llvm::raw_ostream &output,
+ ArrayRef<const Translation *> translations,
+ const MlirTranslateMainConfig &config = {});
+
/// Translate to/from an MLIR module from/to an external representation (e.g.
/// LLVM IR, SPIRV binary, ...). This is the entry point for the implementation
/// of tools like `mlir-translate`. The translation to perform is parsed from
diff --git a/mlir/lib/Tools/mlir-translate/MlirTranslateMain.cpp b/mlir/lib/Tools/mlir-translate/MlirTranslateMain.cpp
index 8472ac0a56e9e..d9db80afbf3fd 100644
--- a/mlir/lib/Tools/mlir-translate/MlirTranslateMain.cpp
+++ b/mlir/lib/Tools/mlir-translate/MlirTranslateMain.cpp
@@ -41,6 +41,101 @@ class ErrorDiagnosticFilter : public ScopedDiagnosticHandler {
// Translate Entry Point
//===----------------------------------------------------------------------===//
+static LogicalResult mlirTranslateMainImpl(
+ std::unique_ptr<llvm::MemoryBuffer> input, llvm::raw_ostream &output,
+ ArrayRef<const Translation *> translations,
+ const MlirTranslateMainConfig &config, TimingScope timing) {
+ if (translations.empty()) {
+ llvm::errs() << "no translation requested\n";
+ return failure();
+ }
+
+ // Processes the memory buffer with a new MLIRContext.
+ auto processBuffer = [&](std::unique_ptr<llvm::MemoryBuffer> ownedBuffer,
+ raw_ostream &os) {
+ // Many of the translations expect a null-terminated buffer while splitting
+ // the buffer does not guarantee null-termination. Make a copy of the buffer
+ // to ensure null-termination.
+ if (!ownedBuffer->getBuffer().ends_with('\0')) {
+ ownedBuffer = llvm::MemoryBuffer::getMemBufferCopy(
+ ownedBuffer->getBuffer(), ownedBuffer->getBufferIdentifier());
+ }
+ // Temporary buffers for chained translation processing.
+ std::string dataIn;
+ std::string dataOut;
+ LogicalResult result = LogicalResult::success();
+
+ for (const auto [index, translationRequested] :
+ llvm::enumerate(translations)) {
+ llvm::raw_ostream *stream;
+ llvm::raw_string_ostream dataStream(dataOut);
+
+ if (index == translations.size() - 1) {
+ // Output last translation to output.
+ stream = &os;
+ } else {
+ // Output translation to temporary data buffer.
+ stream = &dataStream;
+ }
+
+ TimingScope translationTiming =
+ timing.nest(translationRequested->getDescription());
+
+ MLIRContext context;
+ context.allowUnregisteredDialects(
+ config.shouldAllowUnregisteredDialects());
+ context.printOpOnDiagnostic(
+ config.getVerifyDiagnosticsLevel() ==
+ SourceMgrDiagnosticVerifierHandler::Level::None);
+ auto sourceMgr = std::make_shared<llvm::SourceMgr>();
+ sourceMgr->AddNewSourceBuffer(std::move(ownedBuffer), SMLoc());
+
+ if (config.getVerifyDiagnosticsLevel() !=
+ SourceMgrDiagnosticVerifierHandler::Level::None) {
+ // In the diagnostic verification flow, we ignore whether the
+ // translation failed (in most cases, it is expected to fail) and we do
+ // not filter non-error diagnostics even if `errorDiagnosticsOnly` is
+ // set. Instead, we check if the diagnostics were produced as expected.
+ SourceMgrDiagnosticVerifierHandler sourceMgrHandler(
+ *sourceMgr, &context, config.getVerifyDiagnosticsLevel());
+ (void)(*translationRequested)(sourceMgr, os, &context);
+ result = sourceMgrHandler.verify();
+ } else if (config.shouldEmitErrorDiagnosticsOnly()) {
+ SourceMgrDiagnosticHandler sourceMgrHandler(*sourceMgr, &context);
+ ErrorDiagnosticFilter diagnosticFilter(&context);
+ result = (*translationRequested)(sourceMgr, *stream, &context);
+ } else {
+ SourceMgrDiagnosticHandler sourceMgrHandler(*sourceMgr, &context);
+ result = (*translationRequested)(sourceMgr, *stream, &context);
+ }
+ if (failed(result))
+ return result;
+
+ if (index < translations.size() - 1) {
+ // If there are further translations, create a new buffer with the
+ // output data.
+ dataIn = dataOut;
+ dataOut.clear();
+ ownedBuffer = llvm::MemoryBuffer::getMemBuffer(dataIn);
+ }
+ }
+ return result;
+ };
+
+ return splitAndProcessBuffer(std::move(input), processBuffer, output,
+ config.getInputSplitMarker(),
+ config.getOutputSplitMarker());
+}
+
+LogicalResult
+mlir::mlirTranslateMain(std::unique_ptr<llvm::MemoryBuffer> input,
+ llvm::raw_ostream &output,
+ ArrayRef<const Translation *> translations,
+ const MlirTranslateMainConfig &config) {
+ return mlirTranslateMainImpl(std::move(input), output, translations, config,
+ TimingScope());
+}
+
LogicalResult mlir::mlirTranslateMain(int argc, char **argv,
llvm::StringRef toolName) {
@@ -54,7 +149,8 @@ LogicalResult mlir::mlirTranslateMain(int argc, char **argv,
static llvm::cl::opt<bool> allowUnregisteredDialects(
"allow-unregistered-dialect",
- llvm::cl::desc("Allow operation with no registered dialects (discouraged: testing only!)"),
+ llvm::cl::desc("Allow operation with no registered dialects "
+ "(discouraged: testing only!)"),
llvm::cl::init(false));
static llvm::cl::opt<std::string> inputSplitMarker{
@@ -129,77 +225,18 @@ LogicalResult mlir::mlirTranslateMain(int argc, char **argv,
return failure();
}
- // Processes the memory buffer with a new MLIRContext.
- auto processBuffer = [&](std::unique_ptr<llvm::MemoryBuffer> ownedBuffer,
- raw_ostream &os) {
- // Many of the translations expect a null-terminated buffer while splitting
- // the buffer does not guarantee null-termination. Make a copy of the buffer
- // to ensure null-termination.
- if (!ownedBuffer->getBuffer().ends_with('\0')) {
- ownedBuffer = llvm::MemoryBuffer::getMemBufferCopy(
- ownedBuffer->getBuffer(), ownedBuffer->getBufferIdentifier());
- }
- // Temporary buffers for chained translation processing.
- std::string dataIn;
- std::string dataOut;
- LogicalResult result = LogicalResult::success();
-
- for (size_t i = 0, e = translationsRequested.size(); i < e; ++i) {
- llvm::raw_ostream *stream;
- llvm::raw_string_ostream dataStream(dataOut);
-
- if (i == e - 1) {
- // Output last translation to output.
- stream = &os;
- } else {
- // Output translation to temporary data buffer.
- stream = &dataStream;
- }
-
- const Translation *translationRequested = translationsRequested[i];
- TimingScope translationTiming =
- timing.nest(translationRequested->getDescription());
-
- MLIRContext context;
- context.allowUnregisteredDialects(allowUnregisteredDialects);
- context.printOpOnDiagnostic(verifyDiagnostics.getNumOccurrences() == 0);
- auto sourceMgr = std::make_shared<llvm::SourceMgr>();
- sourceMgr->AddNewSourceBuffer(std::move(ownedBuffer), SMLoc());
-
- if (verifyDiagnostics.getNumOccurrences()) {
- // In the diagnostic verification flow, we ignore whether the
- // translation failed (in most cases, it is expected to fail) and we do
- // not filter non-error diagnostics even if `errorDiagnosticsOnly` is
- // set. Instead, we check if the diagnostics were produced as expected.
- SourceMgrDiagnosticVerifierHandler sourceMgrHandler(
- *sourceMgr, &context, verifyDiagnostics);
- (void)(*translationRequested)(sourceMgr, os, &context);
- result = sourceMgrHandler.verify();
- } else if (errorDiagnosticsOnly) {
- SourceMgrDiagnosticHandler sourceMgrHandler(*sourceMgr, &context);
- ErrorDiagnosticFilter diagnosticFilter(&context);
- result = (*translationRequested)(sourceMgr, *stream, &context);
- } else {
- SourceMgrDiagnosticHandler sourceMgrHandler(*sourceMgr, &context);
- result = (*translationRequested)(sourceMgr, *stream, &context);
- }
- if (failed(result))
- return result;
-
- if (i < e - 1) {
- // If there are further translations, create a new buffer with the
- // output data.
- dataIn = dataOut;
- dataOut.clear();
- ownedBuffer = llvm::MemoryBuffer::getMemBuffer(dataIn);
- }
- }
- return result;
- };
-
- if (failed(splitAndProcessBuffer(std::move(input), processBuffer,
- output->os(), inputSplitMarker,
- outputSplitMarker)))
+ MlirTranslateMainConfig config;
+ config.allowUnregisteredDialects(allowUnregisteredDialects)
+ .errorDiagnosticsOnly(errorDiagnosticsOnly)
+ .outputSplitMarker(outputSplitMarker);
+ if (inputSplitMarker.getNumOccurrences())
+ config.splitInputFile(inputSplitMarker);
+ if (verifyDiagnostics.getNumOccurrences())
+ config.verifyDiagnostics(verifyDiagnostics);
+
+ if (failed(mlirTranslateMainImpl(std::move(input), output->os(),
+ translationsRequested, config,
+ std::move(timing))))
return failure();
output->keep();
diff --git a/mlir/unittests/CMakeLists.txt b/mlir/unittests/CMakeLists.txt
index 25514840c0f58..c8b351552dedc 100644
--- a/mlir/unittests/CMakeLists.txt
+++ b/mlir/unittests/CMakeLists.txt
@@ -20,6 +20,7 @@ add_subdirectory(Support)
add_subdirectory(Rewrite)
add_subdirectory(TableGen)
add_subdirectory(Target)
+add_subdirectory(Tools)
add_subdirectory(Transforms)
if(MLIR_ENABLE_EXECUTION_ENGINE)
diff --git a/mlir/unittests/Tools/CMakeLists.txt b/mlir/unittests/Tools/CMakeLists.txt
new file mode 100644
index 0000000000000..85abb92ea5923
--- /dev/null
+++ b/mlir/unittests/Tools/CMakeLists.txt
@@ -0,0 +1 @@
+add_subdirectory(mlir-translate)
diff --git a/mlir/unittests/Tools/mlir-translate/CMakeLists.txt b/mlir/unittests/Tools/mlir-translate/CMakeLists.txt
new file mode 100644
index 0000000000000..05673a0ff938a
--- /dev/null
+++ b/mlir/unittests/Tools/mlir-translate/CMakeLists.txt
@@ -0,0 +1,8 @@
+add_mlir_unittest(MLIRTranslateMainTests
+ MlirTranslateMainTest.cpp
+)
+
+target_link_libraries(MLIRTranslateMainTests
+ PRIVATE
+ MLIRTranslateLib
+)
diff --git a/mlir/unittests/Tools/mlir-translate/MlirTranslateMainTest.cpp b/mlir/unittests/Tools/mlir-translate/MlirTranslateMainTest.cpp
new file mode 100644
index 0000000000000..2456b1b49dd25
--- /dev/null
+++ b/mlir/unittests/Tools/mlir-translate/MlirTranslateMainTest.cpp
@@ -0,0 +1,46 @@
+//===- MlirTranslateMainTest.cpp - mlir-translate tests -------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "mlir/Tools/mlir-translate/MlirTranslateMain.h"
+#include "mlir/Tools/mlir-translate/Translation.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/SourceMgr.h"
+#include "llvm/Support/raw_ostream.h"
+#include "gtest/gtest.h"
+
+using namespace mlir;
+
+namespace {
+TEST(MlirTranslateMainTest, RunsTranslationsWithoutCommandLineParsing) {
+ llvm::cl::opt<std::string> existingOutputOption("o");
+ (void)existingOutputOption;
+
+ auto append = [](StringRef suffix) {
+ return Translation(
+ [suffix](const std::shared_ptr<llvm::SourceMgr> &sourceMgr,
+ llvm::raw_ostream &output, MLIRContext *) {
+ const llvm::MemoryBuffer *input =
+ sourceMgr->getMemoryBuffer(sourceMgr->getMainFileID());
+ output << input->getBuffer() << suffix;
+ return success();
+ },
+ "append text", std::nullopt);
+ };
+
+ Translation first = append(" first");
+ Translation second = append(" second");
+ const Translation *translations[] = {&first, &second};
+ std::string output;
+ llvm::raw_string_ostream outputStream(output);
+
+ EXPECT_TRUE(succeeded(mlirTranslateMain(
+ llvm::MemoryBuffer::getMemBuffer("input"), outputStream, translations)));
+ EXPECT_EQ(output, "input first second");
+}
+} // namespace
More information about the Mlir-commits
mailing list