[llvm-dev] Can we do atomic write to a file by using raw_fd_ostream?

[Tim Northover via llvm-dev]

Hi Jie,


On Sat, 6 Apr 2019 at 01:43, Jie Zhou via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> In a pass I’m using raw_fd_ostream to write a string to a file. Does raw_fd_ostream guarantee
> the write is atomic when I do parallel compilation (currently I’m using -j8)?

raw_fd_ostream doesn't really attempt to solve this problem. All
writes get filtered through the write_impl function which calls POSIX
::write in chunks (large ones -- 1GB). Additionally, individual calls
to << operators may or may not be buffered, and if they are it's with
a fixed size buffer rather than a transaction based system.

According to https://stackoverflow.com/questions/1154446/is-file-append-atomic-in-unix,
those individual calls to ::write are atomic and won't interfere
provided the file has been opened in append mode. So your goal should
be to ensure that each logging event gets implemented as a single call
to ::write.

The most straightforward way would obviously be to do it yourself. In
LLVM you might do your normal writing to a raw_string_ostream and then
call ::write once on that.

Cheers.

Tim.