[llvm-dev] Can we do atomic write to a file by using raw_fd_ostream?

[Rui Ueyama via llvm-dev]

The other solution would be to use ninja instead of make if you can. Ninja
always buffers command output so that error messages from a parallel build
don't interleave. https://ninja-build.org/manual.html#_comparison_to_make

On Sat, Apr 6, 2019 at 4:02 PM Tim Northover via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi Jie,
>
>
> On Sat, 6 Apr 2019 at 01:43, Jie Zhou via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > In a pass I’m using raw_fd_ostream to write a string to a file. Does
> raw_fd_ostream guarantee
> > the write is atomic when I do parallel compilation (currently I’m using
> -j8)?
>
> raw_fd_ostream doesn't really attempt to solve this problem. All
> writes get filtered through the write_impl function which calls POSIX
> ::write in chunks (large ones -- 1GB). Additionally, individual calls
> to << operators may or may not be buffered, and if they are it's with
> a fixed size buffer rather than a transaction based system.
>
> According to
> https://stackoverflow.com/questions/1154446/is-file-append-atomic-in-unix,
> those individual calls to ::write are atomic and won't interfere
> provided the file has been opened in append mode. So your goal should
> be to ensure that each logging event gets implemented as a single call
> to ::write.
>
> The most straightforward way would obviously be to do it yourself. In
> LLVM you might do your normal writing to a raw_string_ostream and then
> call ::write once on that.
>
> Cheers.
>
> Tim.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190408/e01990a9/attachment.html>