[PATCH] D95501: WIP: Support: Add vfs::OutputManager to virtualize compiler outputs

[Duncan P. N. Exon Smith via Phabricator via llvm-commits]

dexonsmith created this revision.
dexonsmith added reviewers: sammccall, erik.pilkington, marcrasi.
Herald added subscribers: ributzka, jfb, hiraditya, mgorny.
dexonsmith requested review of this revision.
Herald added a project: LLVM.

[This is supporting an RFC which I'll send to cfe-dev and llvm-dev in a bit; a follow up patch has initial adoption in clang::CompilerInstance; marked WIP for now.]

Add `OutputManager` to the `llvm::vfs` namespace to virtualize compiler
outputs.

The manager is configured with an `OutputBackend`. This patch contains a
number of backends, including:

- NullOutputBackend, for ignoring all backends.
- OnDiskOutputBackend, for writing to disk (the default), initially based on the logic in `clang::CompilerInstance`.
- InMemoryOutputBackend, for writing to an `InMemoryFileSystem`.
- MirroringOutputBackend, for writing to multiple backends.
- FilteringOutputBackend, for filtering which outputs get written to the underlying backend.

An `OutputBackend` needs to implement `createDestination()`, which
returns a `std::unique_ptr<OutputDestination>`. Typically this is an
associated subclass of `OutputDestination`.

`OutputDestination`s are chained in a linked list (to support
`MirroringOutputBackend`). Each one needs to implement
`storeContentImpl()` (at minimum) in order to store content.
`ContentBuffer` is used to shepherd the content between backends. Some
`OutputDestination`s don't need content to be buffered, and can
implement `takeStreamImpl()` / `storeStreamedContentImpl()` to bypass
it when they're the only destination.

`OutputManager::createOutput()` returns an RAII-managed
`std::unique_ptr<Output>`.

- `takeOS()` returns a `std::unique_ptr<raw_pwrite_stream>` for writing (`getOS()` gives a raw pointer to the stream if it hasn't been taken).
- `close()` commits the file (to disk, in-memory, etc.).
- `erase()` cancels the output, cleaning it up.
- `~Output` effectively calls `erase()` if the output is still "open".

A `ContentBuffer` is initialized with either
`std::unique_ptr<llvm::MemoryBuffer>` or `SmallVectorImpl<char> &&`.
If a destination just needs the bytes, it can call `getBytes()`; if it
wants a `MemoryBuffer`, it can call `takeBuffer()`, which has a variants
depending on whether the destination needs to own the bytes written to.

`OnDiskOutputDestination` has support for writing its bytes to an
mmap'ed region and passing a file-backed buffer on to other
destinations.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D95501

Files:
  llvm/include/llvm/Support/OutputManager.h
  llvm/lib/Support/CMakeLists.txt
  llvm/lib/Support/OutputManager.cpp
  llvm/unittests/Support/CMakeLists.txt
  llvm/unittests/Support/OutputConfigTest.cpp
  llvm/unittests/Support/OutputManagerTest.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D95501.319455.patch
Type: text/x-patch
Size: 119014 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210127/ce53a7a4/attachment.bin>