<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">I’ve been asked about this, but it’s currently beyond my knowledge. Can someone shed some light? Does Clang need fixing for this?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">***<o:p></o:p></p>
<p class="MsoNormal">> >- Does it allow for processes being terminated whilst creating cache entries?<o:p></o:p></p>
<p class="MsoNormal">> <o:p></o:p></p>
<p class="MsoNormal">> A subsequent run will clean up orphaned file locks from a previously crashed or aborted process,<o:p></o:p></p>
<p class="MsoNormal">> but it's not clear if other modules cache files are dealt with.<o:p></o:p></p>
<p class="MsoNormal">> <o:p></o:p></p>
<p class="MsoNormal">> I think the build system should probably include deleting the entire cache directory during a<o:p></o:p></p>
<p class="MsoNormal">> normal "realclean" or perhaps even a "clean" operation as well. If a build reports errors in<o:p></o:p></p>
<p class="MsoNormal">> intermediate files that's typically the first thing I would try, i.e. do a clean and rebuild.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">When we distribute builds we may abort processes at any point during the build. It is essential<o:p></o:p></p>
<p class="MsoNormal">that doing this does not prevent other processes in the build from completing successfully. In<o:p></o:p></p>
<p class="MsoNormal">this scenario failing the build with an error, doing a clean and a rebuild is not an option.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Our requirement us that if two compilation jobs A and B are running simultaneously and will create<o:p></o:p></p>
<p class="MsoNormal">a cache entry C, then:<o:p></o:p></p>
<p class="MsoNormal">1. A and B will arbitrate access to the cache as necessary to create and read C.<o:p></o:p></p>
<p class="MsoNormal">2. Aborting either A or B at any moment (without any opportunity for clean up by the aborted<o:p></o:p></p>
<p class="MsoNormal">process) will not prevent the other from completing successfully.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The important case is when A holds the lock and is writing C but is terminated before completing<o:p></o:p></p>
<p class="MsoNormal">the write. It is essential that C won't be left in a state where B (either waiting on the same<o:p></o:p></p>
<p class="MsoNormal">lock or running at a later date) will think it is valid, try to use and error out.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This can be avoided if the procedure for writing to the cache goes like this:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">1. Take the lock<o:p></o:p></p>
<p class="MsoNormal">2. If C does not exist then:<o:p></o:p></p>
<p class="MsoNormal">2.1. create C.TEMP<o:p></o:p></p>
<p class="MsoNormal">2.2. Rename C.TEMP to C<o:p></o:p></p>
<p class="MsoNormal">3. Release lock<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This guarantees (in the absence of power outages, disk failures etc) that an invalid C will never<o:p></o:p></p>
<p class="MsoNormal">be left for B to fail on.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">There are some minor implementation wrinkles in making the the rename as atomic as possible (on<o:p></o:p></p>
<p class="MsoNormal">Windows C.TEMP and C must be on the same volume so that a copy is not required), but this ought to<o:p></o:p></p>
<div style="mso-element:para-border-div;border:none;border-bottom:dotted windowtext 3.0pt;padding:0in 0in 1.0pt 0in">
<p class="MsoNormal" style="border:none;padding:0in">be straight forward to implement if it hasn't been already.<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-John<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>