[compiler-rt] r326866 - [XRay][compiler-rt] Add APIs for processing logs in memory
Dean Michael Berris via llvm-commits
llvm-commits at lists.llvm.org
Tue Mar 6 18:45:14 PST 2018
Author: dberris
Date: Tue Mar 6 18:45:14 2018
New Revision: 326866
URL: http://llvm.org/viewvc/llvm-project?rev=326866&view=rev
Log:
[XRay][compiler-rt] Add APIs for processing logs in memory
Summary:
This change adds APIs to allow logging implementations to provide a
function for iterating through in-memory buffers (if they hold in-memory
buffers) and a way for users to generically deal with these buffers
in-process. These APIs are:
- __xray_log_set_buffer_iterator(...) and
__xray_log_remove_buffer_iterator(): installs and removes an
iterator function that takes an XRayBuffer and yields the next one.
- __xray_log_process_buffers(...): takes a function pointer that can
take a mode identifier (string) and an XRayBuffer to process this
data as they see fit.
The intent is to have the FDR mode implementation's buffers be
available through this `__xray_log_process_buffers(...)` API, so that
they can be streamed from memory instead of flushed to disk (useful for
getting the data to a network, or doing in-process analysis).
Basic mode logging will not support this mechanism as it's designed to
write the data mostly to disk.
Future implementations will may depend on this API as well, to allow for
programmatically working through the XRay buffers exposed to the
users in some fashion.
Reviewers: eizan, kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D43495
Modified:
compiler-rt/trunk/include/xray/xray_log_interface.h
compiler-rt/trunk/lib/xray/xray_log_interface.cc
compiler-rt/trunk/test/xray/TestCases/Posix/logging-modes.cc
Modified: compiler-rt/trunk/include/xray/xray_log_interface.h
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/include/xray/xray_log_interface.h?rev=326866&r1=326865&r2=326866&view=diff
==============================================================================
--- compiler-rt/trunk/include/xray/xray_log_interface.h (original)
+++ compiler-rt/trunk/include/xray/xray_log_interface.h Tue Mar 6 18:45:14 2018
@@ -72,6 +72,17 @@
/// // deal with the error here, if it is an error.
/// }
///
+/// // Alternatively, we can go through the buffers ourselves without
+/// // relying on the implementations' flushing semantics (if the
+/// // implementation supports exporting this data directly).
+/// auto MyBufferProcessor = +[](const char* mode, XRayBuffer buffer) {
+/// // Check the "mode" to see if it's something we know how to handle...
+/// // and/or do something with an XRayBuffer instance.
+/// };
+/// auto process_status = __xray_log_process_buffers(MyBufferProcessor);
+/// if (process_status != XRayLogFlushStatus::XRAY_LOG_FLUSHED) {
+/// // deal with the error here, if it is an error.
+/// }
///
/// NOTE: Before calling __xray_patch() again, consider re-initializing the
/// implementation first. Some implementations might stay in an "off" state when
@@ -227,6 +238,11 @@ XRayLogRegisterStatus __xray_log_registe
/// does not update the currently installed implementation.
XRayLogRegisterStatus __xray_log_select_mode(const char *Mode);
+/// Returns an identifier for the currently selected XRay mode chosen through
+/// the __xray_log_select_mode(...) function call. Returns nullptr if there is
+/// no currently installed mode.
+const char *__xray_log_get_current_mode();
+
/// This function removes the currently installed implementation. It will also
/// uninstall any handlers that have been previously installed. It does NOT
/// unpatch the instrumentation sleds.
@@ -257,6 +273,54 @@ XRayLogInitStatus __xray_log_finalize();
/// XRayLogFlushStatus for what the return values mean.
XRayLogFlushStatus __xray_log_flushLog();
+/// An XRayBuffer represents a section of memory which can be treated by log
+/// processing functions as bytes stored in the logging implementation's
+/// buffers.
+struct XRayBuffer {
+ const void *Data;
+ size_t Size;
+};
+
+/// Registers an iterator function which takes an XRayBuffer argument, then
+/// returns another XRayBuffer function representing the next buffer. When the
+/// Iterator function returns an empty XRayBuffer (Data = nullptr, Size = 0),
+/// this signifies the end of the buffers.
+///
+/// The first invocation of this Iterator function will always take an empty
+/// XRayBuffer (Data = nullptr, Size = 0).
+void __xray_log_set_buffer_iterator(XRayBuffer (*Iterator)(XRayBuffer));
+
+/// Removes the currently registered buffer iterator function.
+void __xray_log_remove_buffer_iterator();
+
+/// Invokes the provided handler to process data maintained by the logging
+/// handler. This API will be provided raw access to the data available in
+/// memory from the logging implementation. The callback function must:
+///
+/// 1) Not modify the data, to avoid running into undefined behaviour.
+///
+/// 2) Either know the data layout, or treat the data as raw bytes for later
+/// interpretation.
+///
+/// This API is best used in place of the `__xray_log_flushLog()` implementation
+/// above to enable the caller to provide an alternative means of extracting the
+/// data from the XRay implementation.
+///
+/// Implementations MUST then provide:
+///
+/// 1) A function that will return an XRayBuffer. Functions that return an
+/// "empty" XRayBuffer signifies that there are no more buffers to be
+/// processed. This function should be registered through the
+/// `__xray_log_set_buffer_iterator(...)` function.
+///
+/// 2) Its own means of converting data it holds in memory into an XRayBuffer
+/// structure.
+///
+/// See XRayLogFlushStatus for what the return values mean.
+///
+XRayLogFlushStatus __xray_log_process_buffers(void (*Processor)(const char *,
+ XRayBuffer));
+
} // extern "C"
namespace __xray {
Modified: compiler-rt/trunk/lib/xray/xray_log_interface.cc
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/xray/xray_log_interface.cc?rev=326866&r1=326865&r2=326866&view=diff
==============================================================================
--- compiler-rt/trunk/lib/xray/xray_log_interface.cc (original)
+++ compiler-rt/trunk/lib/xray/xray_log_interface.cc Tue Mar 6 18:45:14 2018
@@ -18,9 +18,20 @@
#include "xray/xray_interface.h"
#include "xray_defs.h"
-__sanitizer::SpinMutex XRayImplMutex;
-XRayLogImpl CurrentXRayImpl{nullptr, nullptr, nullptr, nullptr};
-XRayLogImpl *GlobalXRayImpl = nullptr;
+namespace __xray {
+static __sanitizer::SpinMutex XRayImplMutex;
+static XRayLogImpl CurrentXRayImpl{nullptr, nullptr, nullptr, nullptr};
+static XRayLogImpl *GlobalXRayImpl = nullptr;
+
+// This is the default implementation of a buffer iterator, which always yields
+// a null buffer.
+XRayBuffer NullBufferIterator(XRayBuffer) XRAY_NEVER_INSTRUMENT {
+ return {nullptr, 0};
+}
+
+// This is the global function responsible for iterating through given buffers.
+__sanitizer::atomic_uintptr_t XRayBufferIterator{
+ reinterpret_cast<uintptr_t>(&NullBufferIterator)};
// We use a linked list of Mode to XRayLogImpl mappings. This is a linked list
// when it should be a map because we're avoiding having to depend on C++
@@ -31,9 +42,25 @@ struct ModeImpl {
XRayLogImpl Impl;
};
-ModeImpl SentinelModeImpl{
+static ModeImpl SentinelModeImpl{
nullptr, nullptr, {nullptr, nullptr, nullptr, nullptr}};
-ModeImpl *ModeImpls = &SentinelModeImpl;
+static ModeImpl *ModeImpls = &SentinelModeImpl;
+static const ModeImpl *CurrentMode = nullptr;
+
+} // namespace __xray
+
+using namespace __xray;
+
+void __xray_log_set_buffer_iterator(XRayBuffer (*Iterator)(XRayBuffer))
+ XRAY_NEVER_INSTRUMENT {
+ __sanitizer::atomic_store(&__xray::XRayBufferIterator,
+ reinterpret_cast<uintptr_t>(Iterator),
+ __sanitizer::memory_order_release);
+}
+
+void __xray_log_remove_buffer_iterator() XRAY_NEVER_INSTRUMENT {
+ __xray_log_set_buffer_iterator(&NullBufferIterator);
+}
XRayLogRegisterStatus
__xray_log_register_mode(const char *Mode,
@@ -62,6 +89,7 @@ __xray_log_select_mode(const char *Mode)
__sanitizer::SpinMutexLock Guard(&XRayImplMutex);
for (ModeImpl *it = ModeImpls; it != &SentinelModeImpl; it = it->Next) {
if (!__sanitizer::internal_strcmp(Mode, it->Mode)) {
+ CurrentMode = it;
CurrentXRayImpl = it->Impl;
GlobalXRayImpl = &CurrentXRayImpl;
__xray_set_handler(it->Impl.handle_arg0);
@@ -71,11 +99,19 @@ __xray_log_select_mode(const char *Mode)
return XRayLogRegisterStatus::XRAY_MODE_NOT_FOUND;
}
+const char *__xray_log_get_current_mode() XRAY_NEVER_INSTRUMENT {
+ __sanitizer::SpinMutexLock Guard(&XRayImplMutex);
+ if (CurrentMode != nullptr)
+ return CurrentMode->Mode;
+ return nullptr;
+}
+
void __xray_set_log_impl(XRayLogImpl Impl) XRAY_NEVER_INSTRUMENT {
if (Impl.log_init == nullptr || Impl.log_finalize == nullptr ||
Impl.handle_arg0 == nullptr || Impl.flush_log == nullptr) {
__sanitizer::SpinMutexLock Guard(&XRayImplMutex);
GlobalXRayImpl = nullptr;
+ CurrentMode = nullptr;
__xray_remove_handler();
__xray_remove_handler_arg1();
return;
@@ -116,3 +152,20 @@ XRayLogFlushStatus __xray_log_flushLog()
return XRayLogFlushStatus::XRAY_LOG_NOT_FLUSHING;
return GlobalXRayImpl->flush_log();
}
+
+XRayLogFlushStatus __xray_log_process_buffers(
+ void (*Processor)(const char *, XRayBuffer)) XRAY_NEVER_INSTRUMENT {
+ // We want to make sure that there will be no changes to the global state for
+ // the log by synchronising on the XRayBufferIteratorMutex.
+ if (!GlobalXRayImpl)
+ return XRayLogFlushStatus::XRAY_LOG_NOT_FLUSHING;
+ auto Iterator = reinterpret_cast<XRayBuffer (*)(XRayBuffer)>(
+ atomic_load(&XRayBufferIterator, __sanitizer::memory_order_acquire));
+ auto Buffer = (*Iterator)(XRayBuffer{nullptr, 0});
+ auto Mode = CurrentMode ? CurrentMode->Mode : nullptr;
+ while (Buffer.Data != nullptr) {
+ (*Processor)(Mode, Buffer);
+ Buffer = (*Iterator)(Buffer);
+ }
+ return XRayLogFlushStatus::XRAY_LOG_FLUSHED;
+}
Modified: compiler-rt/trunk/test/xray/TestCases/Posix/logging-modes.cc
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/test/xray/TestCases/Posix/logging-modes.cc?rev=326866&r1=326865&r2=326866&view=diff
==============================================================================
--- compiler-rt/trunk/test/xray/TestCases/Posix/logging-modes.cc (original)
+++ compiler-rt/trunk/test/xray/TestCases/Posix/logging-modes.cc Tue Mar 6 18:45:14 2018
@@ -9,6 +9,7 @@
#include "xray/xray_log_interface.h"
#include <cassert>
#include <cstdio>
+#include <string>
[[clang::xray_never_instrument]] void printing_handler(int32_t fid,
XRayEntryType) {
@@ -20,8 +21,19 @@
printing = false;
}
+[[clang::xray_never_instrument]] XRayBuffer next_buffer(XRayBuffer buffer) {
+ static const char data[10] = {};
+ static const XRayBuffer first_and_last{data, 10};
+ if (buffer.Data == nullptr)
+ return first_and_last;
+ if (buffer.Data == first_and_last.Data)
+ return XRayBuffer{nullptr, 0};
+ assert(false && "Invalid buffer provided.");
+}
+
[[clang::xray_never_instrument]] XRayLogInitStatus
printing_init(size_t, size_t, void *, size_t) {
+ __xray_log_set_buffer_iterator(next_buffer);
return XRayLogInitStatus::XRAY_LOG_INITIALIZED;
}
@@ -30,11 +42,16 @@ printing_init(size_t, size_t, void *, si
}
[[clang::xray_never_instrument]] XRayLogFlushStatus printing_flush_log() {
+ __xray_log_remove_buffer_iterator();
return XRayLogFlushStatus::XRAY_LOG_FLUSHED;
}
[[clang::xray_always_instrument]] void callme() { std::printf("called me!\n"); }
+static auto buffer_counter = 0;
+
+void process_buffer(const char *, XRayBuffer) { ++buffer_counter; }
+
static bool unused = [] {
assert(__xray_log_register_mode("custom",
{printing_init, printing_finalize,
@@ -46,6 +63,9 @@ static bool unused = [] {
int main(int argc, char **argv) {
assert(__xray_log_select_mode("custom") ==
XRayLogRegisterStatus::XRAY_REGISTRATION_OK);
+ assert(__xray_log_get_current_mode() != nullptr);
+ std::string current_mode = __xray_log_get_current_mode();
+ assert(current_mode == "custom");
assert(__xray_patch() == XRayPatchingStatus::SUCCESS);
assert(__xray_log_init(0, 0, nullptr, 0) ==
XRayLogInitStatus::XRAY_LOG_INITIALIZED);
@@ -53,6 +73,9 @@ int main(int argc, char **argv) {
callme(); // CHECK: called me!
// CHECK: printing {{.*}}
assert(__xray_log_finalize() == XRayLogInitStatus::XRAY_LOG_FINALIZED);
+ assert(__xray_log_process_buffers(process_buffer) ==
+ XRayLogFlushStatus::XRAY_LOG_FLUSHED);
+ assert(buffer_counter == 1);
assert(__xray_log_flushLog() == XRayLogFlushStatus::XRAY_LOG_FLUSHED);
assert(__xray_log_select_mode("not-found") ==
XRayLogRegisterStatus::XRAY_MODE_NOT_FOUND);
More information about the llvm-commits
mailing list