<div dir="ltr">Thanks, Jan. And one more time, sorry about the inconvenience.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 10, 2019 at 12:47 AM Jan Korous <<a href="mailto:jkorous@apple.com">jkorous@apple.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;">Ok, no worries. I understand the reason for the revert and will rewrite the patch.<br><div><br><div><br><blockquote type="cite"><div>On Oct 9, 2019, at 7:19 AM, Ilya Biryukov <<a href="mailto:ibiryukov@google.com" target="_blank">ibiryukov@google.com</a>> wrote:</div><br><div><div dir="ltr">Fair enough, sorry about that.<div><br></div><div>Nevertheless, I wanted to re-raise concerns about the approach taken in the patch. It seems to assume invariants about VFS that don't necessarily hold for filesystems outside those tested in upstream LLVM.<div>PTAL at the original patch - calling `getRealPath` on the FS and later using real-file-system with the results assumes the passed VFS is an overlay over the physical filesystem.</div><div>A simple unit-test with InMemoryFS would break right away, so the failing case is pretty obvious.</div><div><br></div><div>There's an obvious alternative: just pass VFS to SanitizerBlacklist (in fact, it already gets a source manager, which has a link to the VFS) and use it for file accesses.</div></div><div>I would suggest doing another round of review on this, happy to take part too.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 9, 2019 at 3:55 PM Nico Weber <<a href="mailto:thakis@chromium.org" target="_blank">thakis@chromium.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">FWIW reverting a patch for it breaking some internal system seems like poor form to me. It's really hard to reland in that case. Please make a reduced repro next time.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 9, 2019 at 5:38 AM Ilya Biryukov via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Reverted in r374151.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 9, 2019 at 11:03 AM Ilya Biryukov <<a href="mailto:ibiryukov@google.com" target="_blank">ibiryukov@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Jan,<div><br></div><div>This patch seems to assume VFS will only re-map some paths, but still point into the physical filesystem.</div><div>This may not be the case, e.g. in unit tests.</div><div><br></div><div>This also manages to break some of our internal clang-tidy integrations for obscure reasons.</div><div><br></div><div>Can we instead just pass VFS instance to the SanitizerBlacklist and use relative paths?</div><div><br></div><div>We might also need to revert the patch to unbreak our release, sorry about the inconvenience.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 8, 2019 at 3:11 AM Jan Korous via cfe-commits <<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Author: jkorous<br>
Date: Mon Oct 7 18:13:17 2019<br>
New Revision: 374006<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=374006&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=374006&view=rev</a><br>
Log:<br>
Reland 'Add VFS support for sanitizers' blacklist'<br>
<br>
The original patch broke the test for Windows.<br>
Trying to fix as per Reid's suggestions outlined here:<br>
<a href="https://reviews.llvm.org/rC371663" rel="noreferrer" target="_blank">https://reviews.llvm.org/rC371663</a><br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D67742" rel="noreferrer" target="_blank">https://reviews.llvm.org/D67742</a><br>
<br>
Added:<br>
cfe/trunk/test/CodeGen/Inputs/sanitizer-blacklist-vfsoverlay.yaml<br>
Modified:<br>
cfe/trunk/lib/AST/ASTContext.cpp<br>
cfe/trunk/test/CodeGen/ubsan-blacklist.c<br>
<br>
Modified: cfe/trunk/lib/AST/ASTContext.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/ASTContext.cpp?rev=374006&r1=374005&r2=374006&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/ASTContext.cpp?rev=374006&r1=374005&r2=374006&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/lib/AST/ASTContext.cpp (original)<br>
+++ cfe/trunk/lib/AST/ASTContext.cpp Mon Oct 7 18:13:17 2019<br>
@@ -72,6 +72,7 @@<br>
#include "llvm/ADT/PointerUnion.h"<br>
#include "llvm/ADT/STLExtras.h"<br>
#include "llvm/ADT/SmallPtrSet.h"<br>
+#include "llvm/ADT/SmallString.h"<br>
#include "llvm/ADT/SmallVector.h"<br>
#include "llvm/ADT/StringExtras.h"<br>
#include "llvm/ADT/StringRef.h"<br>
@@ -81,6 +82,7 @@<br>
#include "llvm/Support/Compiler.h"<br>
#include "llvm/Support/ErrorHandling.h"<br>
#include "llvm/Support/MathExtras.h"<br>
+#include "llvm/Support/VirtualFileSystem.h"<br>
#include "llvm/Support/raw_ostream.h"<br>
#include <algorithm><br>
#include <cassert><br>
@@ -826,6 +828,18 @@ static bool isAddrSpaceMapManglingEnable<br>
llvm_unreachable("getAddressSpaceMapMangling() doesn't cover anything.");<br>
}<br>
<br>
+static std::vector<std::string><br>
+getRealPaths(llvm::vfs::FileSystem &VFS, llvm::ArrayRef<std::string> Paths) {<br>
+ std::vector<std::string> Result;<br>
+ llvm::SmallString<128> Buffer;<br>
+ for (const auto &File : Paths) {<br>
+ if (std::error_code EC = VFS.getRealPath(File, Buffer))<br>
+ llvm::report_fatal_error("can't open file '" + File + "': " + EC.message());<br>
+ Result.push_back(Buffer.str());<br>
+ }<br>
+ return Result;<br>
+}<br>
+<br>
ASTContext::ASTContext(LangOptions &LOpts, SourceManager &SM,<br>
IdentifierTable &idents, SelectorTable &sels,<br>
Builtin::Context &builtins)<br>
@@ -833,7 +847,10 @@ ASTContext::ASTContext(LangOptions &LOpt<br>
TemplateSpecializationTypes(this_()),<br>
DependentTemplateSpecializationTypes(this_()),<br>
SubstTemplateTemplateParmPacks(this_()), SourceMgr(SM), LangOpts(LOpts),<br>
- SanitizerBL(new SanitizerBlacklist(LangOpts.SanitizerBlacklistFiles, SM)),<br>
+ SanitizerBL(new SanitizerBlacklist(<br>
+ getRealPaths(SM.getFileManager().getVirtualFileSystem(),<br>
+ LangOpts.SanitizerBlacklistFiles),<br>
+ SM)),<br>
XRayFilter(new XRayFunctionFilter(LangOpts.XRayAlwaysInstrumentFiles,<br>
LangOpts.XRayNeverInstrumentFiles,<br>
LangOpts.XRayAttrListFiles, SM)),<br>
<br>
Added: cfe/trunk/test/CodeGen/Inputs/sanitizer-blacklist-vfsoverlay.yaml<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/Inputs/sanitizer-blacklist-vfsoverlay.yaml?rev=374006&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/Inputs/sanitizer-blacklist-vfsoverlay.yaml?rev=374006&view=auto</a><br>
==============================================================================<br>
--- cfe/trunk/test/CodeGen/Inputs/sanitizer-blacklist-vfsoverlay.yaml (added)<br>
+++ cfe/trunk/test/CodeGen/Inputs/sanitizer-blacklist-vfsoverlay.yaml Mon Oct 7 18:13:17 2019<br>
@@ -0,0 +1,15 @@<br>
+{<br>
+ 'version': 0,<br>
+ 'roots': [<br>
+ { 'name': '@DIR@', 'type': 'directory',<br>
+ 'contents': [<br>
+ { 'name': 'only-virtual-file.blacklist', 'type': 'file',<br>
+ 'external-contents': '@REAL_FILE@'<br>
+ },<br>
+ { 'name': 'invalid-virtual-file.blacklist', 'type': 'file',<br>
+ 'external-contents': '@NONEXISTENT_FILE@'<br>
+ }<br>
+ ]<br>
+ }<br>
+ ]<br>
+}<br>
<br>
Modified: cfe/trunk/test/CodeGen/ubsan-blacklist.c<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/ubsan-blacklist.c?rev=374006&r1=374005&r2=374006&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/ubsan-blacklist.c?rev=374006&r1=374005&r2=374006&view=diff</a><br>
==============================================================================<br>
--- cfe/trunk/test/CodeGen/ubsan-blacklist.c (original)<br>
+++ cfe/trunk/test/CodeGen/ubsan-blacklist.c Mon Oct 7 18:13:17 2019<br>
@@ -5,6 +5,17 @@<br>
// RUN: %clang_cc1 -fsanitize=unsigned-integer-overflow -fsanitize-blacklist=%t-func.blacklist -emit-llvm %s -o - | FileCheck %s --check-prefix=FUNC<br>
// RUN: %clang_cc1 -fsanitize=unsigned-integer-overflow -fsanitize-blacklist=%t-file.blacklist -emit-llvm %s -o - | FileCheck %s --check-prefix=FILE<br>
<br>
+// RUN: rm -f %t-vfsoverlay.yaml<br>
+// RUN: rm -f %t-nonexistent.blacklist<br>
+// RUN: sed -e "s|@DIR@|%/T|g" %S/Inputs/sanitizer-blacklist-vfsoverlay.yaml | sed -e "s|@REAL_FILE@|%/t-func.blacklist|g" | sed -e "s|@NONEXISTENT_FILE@|%/t-nonexistent.blacklist|g" > %t-vfsoverlay.yaml<br>
+// RUN: %clang_cc1 -fsanitize=unsigned-integer-overflow -ivfsoverlay %t-vfsoverlay.yaml -fsanitize-blacklist=%T/only-virtual-file.blacklist -emit-llvm %s -o - | FileCheck %s --check-prefix=FUNC<br>
+<br>
+// RUN: not %clang_cc1 -fsanitize=unsigned-integer-overflow -ivfsoverlay %t-vfsoverlay.yaml -fsanitize-blacklist=%T/invalid-virtual-file.blacklist -emit-llvm %s -o - 2>&1 | FileCheck %s --check-prefix=INVALID-MAPPED-FILE<br>
+// INVALID-MAPPED-FILE: invalid-virtual-file.blacklist': No such file or directory<br>
+<br>
+// RUN: not %clang_cc1 -fsanitize=unsigned-integer-overflow -ivfsoverlay %t-vfsoverlay.yaml -fsanitize-blacklist=%t-nonexistent.blacklist -emit-llvm %s -o - 2>&1 | FileCheck %s --check-prefix=INVALID<br>
+// INVALID: nonexistent.blacklist': No such file or directory<br>
+<br>
unsigned i;<br>
<br>
// DEFAULT: @hash<br>
<br>
<br>
_______________________________________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Regards,</div><div>Ilya Biryukov</div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Regards,</div><div>Ilya Biryukov</div></div></div></div></div>
_______________________________________________<br>
cfe-commits mailing list<br>
<a href="mailto:cfe-commits@lists.llvm.org" target="_blank">cfe-commits@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits</a><br>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Regards,</div><div>Ilya Biryukov</div></div></div></div></div>
</div></blockquote></div><br></div></div></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Regards,</div><div>Ilya Biryukov</div></div></div></div></div>