<div dir="ltr">Ping, especially curious to hear from Richard - I thought we speculated about this issue a few years ago, so hopefully he's got some context.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 1, 2021 at 10:38 AM Ilya Kuteev <<a href="mailto:ilyakuteev@yandex-team.ru">ilyakuteev@yandex-team.ru</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;"><div>I figured out that the problem is in reading PCM files using FileManager.</div><div>clang/lib/Basic/FileManager.cpp:271</div><div>```</div><div>FileEntry &UFE = UniqueRealFiles[Status.getUniqueID()];</div><div>```</div><div>If I remove this cache everything is OK.</div><div>Seems like inode is shared between deleted/edited and newly created PCM file.</div><div>If it happens FileManager returns a FileEntryRef to an invalid file.</div><div><br></div><div>How can I fix this issue? Maybe we need to remove UniqueRealFiles cache</div><div>for PCM files, or make it work with this case?</div><div><br></div>
<div><br><blockquote type="cite"><div>1 марта 2021 г., в 20:13, David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> написал(а):</div><br><div><div dir="ltr">Can't say any of this sounds especially familiar to me - cc'd a few folks who might have some context.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 19, 2021 at 6:28 AM Ilya Kuteev via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="box-sizing:inherit;margin-left:32px;padding:4px 8px 0px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">Hello everyone!</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">I've ran into a strange problem using clang-modules on Linux. I have a Linux machine running Ubuntu 18.04.4 LTS.</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">When I put my ModuleCache on RAM everything is OK</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">When I put ModuleCache on SSD/HDD, very frequently I'm getting different errors with pcm files, like below:</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">```</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">fatal error: module file '/home/ilyakuteev/module_cache/2/1721172304169885905/3OA4HVRBFA8AJ/Base-5W2XL4QH4QAH.pcm' is out of date and needs to be rebuilt: signature mismatch</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">```</pre><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible"><br></pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">```</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">fatal error: malformed or corrupted AST file: 'Unexpected end of file reading 4154469704 of 4738416 bytes'</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">note: after modifying system headers, please delete the module cache at '/home/ilyakuteev/module_cache/1721172304169885905/37WO4K6LHZNB5'</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">fatal error: error in backend: Invalid abbrev number</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">```</pre><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible"><br></pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">```</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">fatal error: malformed or corrupted AST file: 'declaration ID out-of-range for AST file'</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">note: after modifying system headers, please delete the module cache at '/home/ilyakuteev/module_cache/3/1721172304169885905/1907ERGA34IRD'</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">```</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible"><br></pre><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">I’ve done some research and it looks like the failure is inside single clang process, clang writes pcm down on disk, then reads it and gets a malformed file back, or some other problem with reading on-disk pcm.</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible"><br></pre><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">I've found a patch <a href="https://reviews.llvm.org/D22636" target="_blank">https://reviews.llvm.org/D22636</a> and it looks like it fixes my problems, but I don’t want to use it, cause it just rebuilts "owned" module if its AST is corrupted and looks like retry of operation, not a real fix. Also this fix introduces some unstable behavior.</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible"><br></pre><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">My questions:</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">1) Is this a known problem?</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">2) How can I safely hot-fix it before it will be fixed in clang?</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">3) Do I need to provide more logs for this issue to be fixed?</pre><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible"><br></pre><pre style="box-sizing:inherit;margin-top:0px;margin-bottom:0px;padding:0px 4px;font-size:inherit;line-height:inherit;font-variant-ligatures:contextual;white-space:pre-wrap;word-break:normal;font-family:inherit;border-width:0px;border-radius:0px;background-color:transparent;color:inherit;overflow:visible">Thanks</pre></div><div style="box-sizing:inherit;margin-left:32px;padding:0px 8px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace;font-size:13px;font-variant-ligatures:common-ligatures;background-color:rgb(255,255,255)"></div>
<br></div>_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote></div>
</div></blockquote></div><br></div></blockquote></div>