[llvm] [Offload] Add GenericProfilerTy abstraction and APITypes extensions (upstream OMPT device tracing 1/n) (PR #214340)
Jan Patrick Lehr via llvm-commits
llvm-commits at lists.llvm.org
Wed Aug 5 13:56:44 PDT 2026
https://github.com/jplehr created https://github.com/llvm/llvm-project/pull/214340
This is the first PR for a set of patches that implement a port of the downstream AMD implementation for OMPT device tracing support.
A draft PR with all commits is in https://github.com/llvm/llvm-project/pull/200165 for reference.
In this PR:
Introduce a GenericProfilerTy base class that decouples the plugin layer from OMPT. This allows profiling/tracing backends to be implemented independently of the plugin code.
Key changes:
- Add GenericProfiler.h/cpp with virtual hooks for device lifecycle events (init, deinit, loadBinary), kernel launch, data alloc/delete, kernel completion, and data transfer timing
- Add ProfTimerTy RAII timer for scoped alloc/delete measurements
- Add weak getProfilerToAttach() factory (overridable by OMPT)
- Add ExecAsync and ProfilerData fields to __tgt_async_info
- Replace direct OMPT callback invocations in PluginInterface with profiler hook calls (handleInit, handleDeinit, handleLoadBinary, handlePreKernelLaunch)
- Add getDeviceTimeStamp() virtual to GenericDeviceTy
- Add Profiler member and getProfiler() to GenericPluginTy
- Suppress ProfilerData during KLE upload to avoid spurious traces
- Add sync path in AsyncInfoWrapperTy::finalize for !ExecAsync
AI use: The port was created using Cursor and Claude, while the original implementation was done w/o AI-assistance.
>From 4f020a27aa7aad0cb0ede4566ee1b364b2baaa51 Mon Sep 17 00:00:00 2001
From: JP Lehr <JanPatrick.Lehr at amd.com>
Date: Thu, 2 Apr 2026 07:27:58 -0500
Subject: [PATCH] [Offload] Add GenericProfilerTy abstraction and APITypes
extensions
Introduce GenericProfilerTy alongside the existing OMPT callback dispatch.
The weak profiler factory returns a no-op implementation, so the new hooks
are silent while the established callback path continues to handle OMPT
device events.
Co-Authored-By: Dhruva Chakrabarti <dhruva.chakrabarti at amd.com>
Co-Authored-By: Michael Halkenhauser <michaelgerald.halkenhauser at amd.com>
Co-Authored-By: Claude <noreply at anthropic.com>
---
offload/include/Shared/APITypes.h | 5 +
offload/plugins-nextgen/common/CMakeLists.txt | 1 +
.../common/include/GenericProfiler.h | 182 ++++++++++++++++++
.../common/include/PluginInterface.h | 25 ++-
.../common/src/GenericProfiler.cpp | 36 ++++
.../common/src/PluginInterface.cpp | 26 ++-
6 files changed, 272 insertions(+), 3 deletions(-)
create mode 100644 offload/plugins-nextgen/common/include/GenericProfiler.h
create mode 100644 offload/plugins-nextgen/common/src/GenericProfiler.cpp
diff --git a/offload/include/Shared/APITypes.h b/offload/include/Shared/APITypes.h
index 47d8c49bf7ef2..527aa8e617304 100644
--- a/offload/include/Shared/APITypes.h
+++ b/offload/include/Shared/APITypes.h
@@ -85,6 +85,11 @@ struct __tgt_async_info {
/// ensure it is a valid location while the transfer to the device is
/// happening.
KernelLaunchEnvironmentTy KernelLaunchEnvironment;
+
+ /// Opaque handle for profiler-specific data (e.g., OMPT trace record info).
+ /// Owned by the profiler; the runtime threads this pointer through the plugin
+ /// layer to associate async operations with trace records.
+ void *ProfilerData = nullptr;
};
/// This struct contains all of the arguments to a target kernel region launch.
diff --git a/offload/plugins-nextgen/common/CMakeLists.txt b/offload/plugins-nextgen/common/CMakeLists.txt
index 6ad20796c3ca4..19ce102309a72 100644
--- a/offload/plugins-nextgen/common/CMakeLists.txt
+++ b/offload/plugins-nextgen/common/CMakeLists.txt
@@ -11,6 +11,7 @@ add_public_tablegen_target(PluginErrcodes)
# don't want to export `PluginInterface` while `add_llvm_library` requires that.
add_library(PluginCommon OBJECT
src/PluginInterface.cpp
+ src/GenericProfiler.cpp
src/GlobalHandler.cpp
src/JIT.cpp
src/RecordReplay.cpp
diff --git a/offload/plugins-nextgen/common/include/GenericProfiler.h b/offload/plugins-nextgen/common/include/GenericProfiler.h
new file mode 100644
index 0000000000000..ea3c3766554c8
--- /dev/null
+++ b/offload/plugins-nextgen/common/include/GenericProfiler.h
@@ -0,0 +1,182 @@
+//===- GenericProfiler.h - GenericProfiler interface for use in Plugins ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// The GenericProfiler interface allows to implement profiler logic for various
+// backends, such as OMPT or other tracing mechanisms.
+// This enables the plugins to be agnostic of the actual high-level language
+// that is implemented.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef OFFLOAD_PLUGINS_NEXTGEN_COMMON_INCLUDE_GENERICPROFILER_H
+#define OFFLOAD_PLUGINS_NEXTGEN_COMMON_INCLUDE_GENERICPROFILER_H
+
+#include "Shared/APITypes.h"
+
+#include <cstdint>
+#include <functional>
+#include <tuple>
+
+namespace llvm {
+namespace omp {
+namespace target {
+namespace plugin {
+
+struct GenericDeviceTy;
+struct GenericPluginTy;
+class GenericProfilerTy;
+
+template <typename FunT, typename... ArgsT, size_t... IdxSequence>
+void callViaIndexSeq(FunT F, GenericProfilerTy *P, uint64_t StartNanos,
+ uint64_t EndNanos, std::tuple<ArgsT...> Args,
+ std::index_sequence<IdxSequence...>) {
+ F(P, StartNanos, EndNanos, std::get<IdxSequence>(Args)...);
+}
+
+template <typename FunT, typename... ArgsT>
+void callViaUnpack(FunT F, GenericProfilerTy *P, uint64_t StartNanos,
+ uint64_t EndNanos, std::tuple<ArgsT...> Tup) {
+ callViaIndexSeq(F, P, StartNanos, EndNanos, Tup,
+ std::index_sequence_for<ArgsT...>{});
+}
+
+/// Abstraction layer to implement different profiler backends.
+///
+/// The plugins call into the GenericProfilerTy to handle the specific events
+/// with whatever specific backend was instantiated. For now, the supported
+/// backends are limited to an OMPT implementation.
+class GenericProfilerTy {
+public:
+ GenericProfilerTy() = default;
+ virtual ~GenericProfilerTy() = default;
+
+ /// Obtain a pointer to profiler-specific data, if any.
+ virtual void *getProfilerSpecificData() { return nullptr; }
+
+ virtual bool isProfilingEnabled() { return false; }
+
+ /// Set the factors which are used to interpolate the device clock compared to
+ /// the host clock. This follows a simple linear interpolation: Slope * <time>
+ /// + Offset.
+ void setTimeConversionFactors(double Slope, double Offset) {
+ HostToDeviceSlope = Slope;
+ HostToDeviceOffset = Offset;
+ setTimeConversionFactorsImpl(HostToDeviceSlope, HostToDeviceOffset);
+ }
+
+ /// Hook that is called when the plugin is initialized.
+ virtual void handleInit(GenericDeviceTy *Device, GenericPluginTy *Plugin) {}
+
+ /// Hook that is called when the plugin is de-initialized.
+ virtual void handleDeinit(GenericDeviceTy *Device, GenericPluginTy *Plugin) {}
+
+ /// Hook that is called when the device image is loaded.
+ virtual void handleLoadBinary(GenericDeviceTy *Device,
+ GenericPluginTy *Plugin,
+ const StringRef InputTgtImage) {}
+
+ /// Hook that is called when memory is allocated on the device.
+ virtual void handleDataAlloc(uint64_t StartNanos, uint64_t EndNanos,
+ void *HostPtr, uint64_t Size, void *Data) {}
+
+ /// Hook that is called when memory is freed on the device.
+ virtual void handleDataDelete(uint64_t StartNanos, uint64_t EndNanos,
+ void *TgtPtr, void *Data) {}
+
+ /// Hook that is called before launching a kernel.
+ virtual void handlePreKernelLaunch(GenericDeviceTy *Device,
+ uint32_t NumBlocks[3],
+ __tgt_async_info *AI) {}
+
+ /// Hook that is called when the kernel is finished to extract the specific
+ /// timing info for that kernel execution.
+ virtual void handleKernelCompletion(uint64_t StartNanos, uint64_t EndNanos,
+ void *Data) {}
+
+ /// Hook that is called when a data transfer happens to extract timing info
+ /// for that transfer.
+ virtual void handleDataTransfer(uint64_t StartNanos, uint64_t EndNanos,
+ void *Data) {}
+
+ /// Allow factors for time conversion between host and device.
+ virtual void setTimeConversionFactorsImpl(double Slope, double Offset) {}
+
+ /// RAII style timer that measures the elapsed time between construction and
+ /// destruction, then invokes a callback with the profiler, start/end times,
+ /// and any captured arguments.
+ template <typename FnT, typename... ArgsT> class ProfTimerTy {
+ public:
+ ProfTimerTy(FnT &&F, GenericProfilerTy *P, GenericDeviceTy *D, ArgsT... As)
+ : Fun(F), Prof(P), Dev(D), Args(As...) {
+ assert(Prof && "GenericProfilerTy is null");
+ assert(Dev && "GenericDeviceTy is null");
+ if (Prof)
+ StartTime = Prof->getDeviceTimeStamp(Dev);
+ }
+
+ ~ProfTimerTy() {
+ assert(Prof && "GenericProfilerTy is null");
+ assert(Dev && "GenericDeviceTy is null");
+ if (Prof) {
+ uint64_t EndTime = Prof->getDeviceTimeStamp(Dev);
+ callViaUnpack(Fun, Prof, StartTime, EndTime, Args);
+ }
+ }
+
+ private:
+ FnT Fun;
+ GenericProfilerTy *Prof;
+ GenericDeviceTy *Dev;
+ uint64_t StartTime = 0;
+ std::tuple<ArgsT...> Args;
+ };
+
+ template <typename FnT, typename... ArgsT>
+ ProfTimerTy(FnT &&, GenericProfilerTy *, ArgsT...)
+ -> ProfTimerTy<FnT, ArgsT...>;
+
+ template <typename FnT, typename... ArgsT> friend class ProfTimerTy;
+
+ /// Returns an RAII style timer, which will handle data allocation timing.
+ [[nodiscard]] auto getScopedDataAllocTimer(GenericDeviceTy *Dev,
+ void *HostPtr, uint64_t Size,
+ void *ProfData = nullptr) {
+ return ProfTimerTy(
+ [](GenericProfilerTy *P, auto... args) {
+ assert(P && "P was null");
+ P->handleDataAlloc(args...);
+ },
+ this, Dev, HostPtr, Size, ProfData);
+ }
+
+ /// Returns an RAII style timer, which will handle data deletion timing.
+ [[nodiscard]] auto getScopedDataDeleteTimer(GenericDeviceTy *Dev,
+ void *TgtPtr,
+ void *ProfData = nullptr) {
+ return ProfTimerTy(
+ [](GenericProfilerTy *P, auto... args) {
+ assert(P && "P was null");
+ P->handleDataDelete(args...);
+ },
+ this, Dev, TgtPtr, ProfData);
+ }
+
+protected:
+ double HostToDeviceSlope = 1.0;
+ double HostToDeviceOffset = .0;
+
+private:
+ /// Vendor-specific implementation to obtain device time.
+ uint64_t getDeviceTimeStamp(GenericDeviceTy *D);
+};
+} // namespace plugin
+} // namespace target
+} // namespace omp
+} // namespace llvm
+
+#endif // OFFLOAD_PLUGINS_NEXTGEN_COMMON_INCLUDE_GENERICPROFILER_H
diff --git a/offload/plugins-nextgen/common/include/PluginInterface.h b/offload/plugins-nextgen/common/include/PluginInterface.h
index b3675e5a8700f..b6f19cd503137 100644
--- a/offload/plugins-nextgen/common/include/PluginInterface.h
+++ b/offload/plugins-nextgen/common/include/PluginInterface.h
@@ -29,6 +29,7 @@
#include "Shared/Requirements.h"
#include "Shared/Utils.h"
+#include "GenericProfiler.h"
#include "GlobalHandler.h"
#include "JIT.h"
#include "MemoryManager.h"
@@ -57,6 +58,12 @@
using namespace llvm::offload::debug;
+/// Factory function for creating a profiler. The default (weak) implementation
+/// returns a no-op GenericProfilerTy. When OMPT is enabled, a strong override
+/// returns an OmptProfilerTy instance.
+std::unique_ptr<llvm::omp::target::plugin::GenericProfilerTy>
+getProfilerToAttach();
+
namespace llvm {
namespace omp {
namespace target {
@@ -1191,6 +1198,11 @@ struct GenericDeviceTy : public DeviceAllocatorTy {
uint32_t getDebugKind() const { return OMPX_DebugKind; }
virtual uint64_t getClockFrequency() const { return CLOCKS_PER_SEC; }
+ /// Get a device-specific timestamp in nanoseconds, used by the profiler
+ /// for timing device operations. Subclasses should override this to provide
+ /// hardware-accurate timestamps (e.g., via HSA system info).
+ virtual uint64_t getDeviceTimeStamp() { return 0; }
+
/// Get target compute unit kind (e.g., sm_80, or gfx908).
virtual std::string getComputeUnitKind() const { return "unknown"; }
@@ -1469,7 +1481,8 @@ struct GenericPluginTy {
/// Construct a plugin instance.
GenericPluginTy(Triple::ArchType TA)
- : GlobalHandler(nullptr), JIT(TA), RPCServer(nullptr) {}
+ : GlobalHandler(nullptr), JIT(TA), RPCServer(nullptr),
+ Profiler(getProfilerToAttach()) {}
virtual ~GenericPluginTy() {}
@@ -1560,7 +1573,12 @@ struct GenericPluginTy {
/// Tear down any target-specific doorbell resources.
virtual Error deinitRPCDoorbell() { return Plugin::success(); }
- /// Get a reference to the record and replay interface for the plugin.
+ /// Get a pointer to the profiler attached to this plugin.
+ GenericProfilerTy *getProfiler() {
+ assert(Profiler && "Profiler not initialized");
+ return Profiler.get();
+ }
+
/// Initialize a device within the plugin.
Error initDevice(int32_t DeviceId);
@@ -1842,6 +1860,9 @@ struct GenericPluginTy {
/// The interface between the plugin and the GPU for host services.
RPCServerTy *RPCServer;
+
+ /// The profiler backend attached to this plugin (e.g., OMPT).
+ std::unique_ptr<GenericProfilerTy> Profiler;
};
/// Auxiliary interface class for GenericDeviceResourceManagerTy. This class
diff --git a/offload/plugins-nextgen/common/src/GenericProfiler.cpp b/offload/plugins-nextgen/common/src/GenericProfiler.cpp
new file mode 100644
index 0000000000000..350a0cb5e378a
--- /dev/null
+++ b/offload/plugins-nextgen/common/src/GenericProfiler.cpp
@@ -0,0 +1,36 @@
+//===- GenericProfiler.cpp - GenericProfiler implementation ---------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+//===----------------------------------------------------------------------===//
+
+#include "GenericProfiler.h"
+#include "PluginInterface.h"
+
+#include <cstdint>
+#include <memory>
+
+__attribute__((weak))
+std::unique_ptr<llvm::omp::target::plugin::GenericProfilerTy>
+getProfilerToAttach() {
+ return std::make_unique<llvm::omp::target::plugin::GenericProfilerTy>();
+}
+
+namespace llvm {
+namespace omp {
+namespace target {
+namespace plugin {
+
+uint64_t GenericProfilerTy::getDeviceTimeStamp(GenericDeviceTy *D) {
+ if (D)
+ return D->getDeviceTimeStamp();
+ return 0;
+}
+} // namespace plugin
+} // namespace target
+} // namespace omp
+} // namespace llvm
diff --git a/offload/plugins-nextgen/common/src/PluginInterface.cpp b/offload/plugins-nextgen/common/src/PluginInterface.cpp
index e539cbfc55324..0d5319b62f17d 100644
--- a/offload/plugins-nextgen/common/src/PluginInterface.cpp
+++ b/offload/plugins-nextgen/common/src/PluginInterface.cpp
@@ -26,6 +26,8 @@
#include "omp-tools.h"
#endif
+#include "GenericProfiler.h"
+
#include "llvm/Bitcode/BitcodeReader.h"
#include "llvm/Frontend/OpenMP/OMPConstants.h"
#include "llvm/Support/Error.h"
@@ -47,7 +49,9 @@ using namespace llvm::offload::debug;
AsyncInfoWrapperTy::AsyncInfoWrapperTy(GenericDeviceTy &Device,
__tgt_async_info *AsyncInfoPtr)
: Device(Device),
- AsyncInfoPtr(AsyncInfoPtr ? AsyncInfoPtr : &LocalAsyncInfo) {}
+ AsyncInfoPtr(AsyncInfoPtr ? AsyncInfoPtr : &LocalAsyncInfo) {
+ this->AsyncInfoPtr->ProfilerData = nullptr;
+}
Error AsyncInfoWrapperTy::synchronize() {
assert(AsyncInfoPtr && "AsyncInfoWrapperTy already finalized");
@@ -168,9 +172,15 @@ GenericKernelTy::getKernelLaunchEnvironment(
DPxPTR(&LocalKLE), DPxPTR(*AllocOrErr),
sizeof(KernelLaunchEnvironmentTy));
+ // Temporarily suppress ProfilerData so the KLE upload is not traced as
+ // a user data operation.
+ __tgt_async_info *AI = AsyncInfoWrapper;
+ void *SavedProfilerData = AI->ProfilerData;
+ AI->ProfilerData = nullptr;
auto Err = GenericDevice.dataSubmit(*AllocOrErr, &LocalKLE,
sizeof(KernelLaunchEnvironmentTy),
AsyncInfoWrapper);
+ AI->ProfilerData = SavedProfilerData;
if (Err)
return Err;
return static_cast<KernelLaunchEnvironmentTy *>(*AllocOrErr);
@@ -331,6 +341,9 @@ Error GenericKernelTy::launch(GenericDeviceTy &GenericDevice, void **ArgPtrs,
RRHandle = *RRHandleOrErr;
}
+ GenericDevice.Plugin.getProfiler()->handlePreKernelLaunch(
+ &GenericDevice, EffectiveNumBlocks, AsyncInfoWrapper);
+
if (auto Err = launchImpl(GenericDevice, EffectiveNumThreads,
EffectiveNumBlocks, DynBlockMemConf.NativeSize,
KernelArgs, LaunchParams, AsyncInfoWrapper))
@@ -567,6 +580,8 @@ Error GenericDeviceTy::init(GenericPluginTy &Plugin) {
}
#endif
+ Plugin.getProfiler()->handleInit(this, &Plugin);
+
// Read and reinitialize the envars that depend on the device initialization.
// Notice these two envars may change the stack size and heap size of the
// device, so they need the device properly initialized.
@@ -668,6 +683,8 @@ Error GenericDeviceTy::deinit(GenericPluginTy &Plugin) {
}
#endif
+ Plugin.getProfiler()->handleDeinit(this, &Plugin);
+
return deinitImpl();
}
Expected<DeviceImageTy *> GenericDeviceTy::loadBinary(GenericPluginTy &Plugin,
@@ -725,6 +742,8 @@ Expected<DeviceImageTy *> GenericDeviceTy::loadBinary(GenericPluginTy &Plugin,
}
#endif
+ Plugin.getProfiler()->handleLoadBinary(this, &Plugin, InputTgtImage);
+
// Call any global constructors present on the device.
if (auto Err = callGlobalConstructors(Plugin, *Image))
return std::move(Err);
@@ -1006,6 +1025,9 @@ Error GenericDeviceTy::getDeviceMemorySize(uint64_t &DSize) {
Expected<void *> GenericDeviceTy::dataAlloc(int64_t Size, void *HostPtr,
TargetAllocTy Kind,
size_t Alignment) {
+ auto ProfTimer =
+ Plugin.getProfiler()->getScopedDataAllocTimer(this, HostPtr, Size);
+
void *Alloc = nullptr;
// TODO Check alignment.
@@ -1073,6 +1095,8 @@ Expected<void *> GenericDeviceTy::dataAlloc(int64_t Size, void *HostPtr,
}
Error GenericDeviceTy::dataDelete(void *TgtPtr, TargetAllocTy Kind) {
+ auto ProfTimer = Plugin.getProfiler()->getScopedDataDeleteTimer(this, TgtPtr);
+
// Free is a noop when recording or replaying.
if (RecordReplay && RecordReplay->isRecordingOrReplaying())
return RecordReplay->deallocate(TgtPtr);
More information about the llvm-commits
mailing list