[PATCH] D148417: [compiler-rt][profiling] Add an incremental buffer writing mode to libprofile

[Jessica Paquette via Phabricator via llvm-commits]

paquette created this revision.
paquette added reviewers: aemerson, jroelofs, fhahn, zjaffal, nilanjana_basu, phosek, gulfem.
Herald added subscribers: Enna1, StephenFan, dberris.
Herald added a project: All.
paquette requested review of this revision.

In environments with tight memory constraints, users may not be able to actually allocate a buffer large enough to fit all of their program's profiling data.

In this case, a workflow like this would be helpful:

  while (!__llvm_profile_incremental_writer_done()) {
    char Buffer[BUFFER_SIZE];
    if (__llvm_profile_write_buffer_incremental(Buffer, BUFFER_SIZE))
      return -1;
    if (putBufferSomewhere(Buffer, BUFFER_SIZE))
      return -1;
  }

That is, make a smaller buffer, fill it with `BUFFER_SIZE` bytes of data, transfer that somewhere else, and repeat the process until all of the profiling data has been transferred out.

This patch adds the state and APIs necessary to make the above workflow possible.

The added APIs are as follows:

- `int __llvm_profile_init_incremental_writer_state(void)` - Initializes internal structures to remember where we are in the process of writing profiling data
- `__llvm_profile_write_buffer_incremental(char *Buffer, uint64_t WriteSizeInBytes)` - Writes a certain number of bytes into a buffer
- `int __llvm_profile_incremental_writer_done(void)` - Returns whether or not all profiling bytes have been written

The added state is a struct (`IncrementalProfileWriterState`) that keeps track of

- How many sources we need to write data from
- What those sources are
- Which source we're writing from, and where in it we should write from

The struct reuses the existing `ProfDataIOVec` structure used throughout the rest of libprofile.

Currently, binary ID writing is not supported, since it seems very platform-specific. This will emit a warning on platforms that enable binary ID writing.

This patch also adds a couple refactors/error message improvements:

- Factor out some of the header construction code in `lprofWriteDataImpl`
- Add a `PROF_ERR_INTERNAL` to communicate when an error is likely not a user error, but rather a bug of some sort


https://reviews.llvm.org/D148417

Files:
  clang/docs/SourceBasedCodeCoverage.rst
  compiler-rt/lib/profile/InstrProfiling.h
  compiler-rt/lib/profile/InstrProfilingBuffer.c
  compiler-rt/lib/profile/InstrProfilingInternal.h
  compiler-rt/lib/profile/InstrProfilingPort.h
  compiler-rt/lib/profile/InstrProfilingWriter.c
  compiler-rt/test/profile/IncrementalWriteMode/Inputs/buffer-to-file.c
  compiler-rt/test/profile/IncrementalWriteMode/Inputs/write-n-bytes-and-compare-against-whole-buffer-write.c
  compiler-rt/test/profile/IncrementalWriteMode/compare-against-whole-buffer-write.c
  compiler-rt/test/profile/IncrementalWriteMode/construct-buffer.c
  compiler-rt/test/profile/IncrementalWriteMode/debug-info-correlate.c
  compiler-rt/test/profile/IncrementalWriteMode/error-0-byte-write.c
  compiler-rt/test/profile/IncrementalWriteMode/error-exhausted-buffer.c
  compiler-rt/test/profile/IncrementalWriteMode/error-uninitialized-state.c
  compiler-rt/test/profile/IncrementalWriteMode/reinitialize-state.c

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D148417.513856.patch
Type: text/x-patch
Size: 39448 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230415/35f00d19/attachment-0001.bin>