[cfe-dev] C++ modules questions

[Richard Smith]

David, you've looked into this more recently than I have; can you
remember more details about our lock file mechanism?

On Wed, Jan 21, 2015 at 11:25 AM, Thompson, John
<John_Thompson at playstation.sony.com> wrote:
> Our distributed build tool team is asking the following questions regarding
> C++ Modules.  [My guesses in brackets.  Please correct.]
>
> - Is the module creation mechanism (that fills the module cache)
> multi-process safe?
>
> [yes]

Yes, it should be,  in the sense that you should never see an invalid
PCM file. But IIRC it's racy and may perform redundant rebuilds. And
it's slow (see below).

> - If two or more compiler instances both try to create the same entry how is
> this arbitrated?
>
> [O/S locks]

A lock file is used, and the waiting processes poll the filesystem
waiting for the resulting PCM file to appear. This polling starts by
waiting 1s and has exponential backoff. This leads to very high
overheads if modules need to be rebuilt, but no-one has fixed it to
use something more reasonable yet. We assume that file renames within
the module cache atomically create a complete destination file.

> - Does it allow for processes being terminated whilst creating cache
> entries?  [I.e. are incomplete files cleaned up?]
>
> [I don't know]

IIUC, no. Processes clean up after themselves, but there isn't a
mechanism for processes to clean up after other processes that died.