<div dir="ltr">> <span style="font-size:12.8px">Also, it would be awfully nice if -fsanitize=address were the only flag necessary to add to the build to make everything work. Requiring users to add another flag isn't particularly kind.</span><div><span style="font-size:12.8px"><br></span></div><div><span style="font-size:12.8px">It may not be particularly kind, but replacing your STL is something we might want consent to do. Especially if it has all these complexities.</span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 15, 2016 at 1:47 PM, Craig, Ben via cfe-dev <span dir="ltr"><<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5"><br>
<br>
On 8/15/2016 2:27 PM, Jonathan Roelofs via cfe-dev wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
On 8/15/16 12:50 PM, Hal Finkel wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: "Jonathan Roelofs" <<a href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a>><br>
To: "Evgenii Stepanov" <<a href="mailto:eugenis@google.com" target="_blank">eugenis@google.com</a>><br>
Cc: "Hal Finkel" <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>>, "Eric Fiselier" <<a href="mailto:eric@efcs.ca" target="_blank">eric@efcs.ca</a>>, "clang developer list" <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>>,<br>
"Chandler Carruth" <<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>>, "Kostya Serebryany" <<a href="mailto:kcc@google.com" target="_blank">kcc@google.com</a>><br>
Sent: Monday, August 15, 2016 1:37:11 PM<br>
Subject: Re: [cfe-dev] Making MSAN Easier to Use: Providing a Sanitized Libc++<br>
<br>
<br>
<br>
On 8/15/16 12:14 PM, Evgenii Stepanov wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Mon, Aug 15, 2016 at 7:24 AM, Jonathan Roelofs<br>
<<a href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
On 8/14/16 7:31 PM, Hal Finkel wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
----- Original Message -----<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
From: "Jonathan Roelofs via cfe-dev" <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>><br>
To:<br>
"Eric Fiselier" <<a href="mailto:eric@efcs.ca" target="_blank">eric@efcs.ca</a>>, "clang developer list"<br>
<<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>>, "Chandler Carruth"<br>
<<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>>,<br>
"Kostya Serebryany" <<a href="mailto:kcc@google.com" target="_blank">kcc@google.com</a>>, "Evgenii Stepanov"<br>
<<a href="mailto:eugenis@google.com" target="_blank">eugenis@google.com</a>> Sent: Sunday, August 14, 2016 7:07:00 PM<br>
Subject: Re: [cfe-dev] Making MSAN Easier to Use: Providing a<br>
Sanitized Libc++<br>
<br>
<br>
<br>
On 8/14/16 4:05 PM, Eric Fiselier via cfe-dev wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Sanitizers such as MSAN require the entire program to be<br>
instrumented, anything less leads to plenty of false positives.<br>
Unfortunately this can be difficult to achieve, especially for<br>
the C and C++ standard libraries. To work around this the<br>
sanitizers provide interceptors for common C functions, but the<br>
same solution doesn't work as well for the C++ STL. Instead<br>
users<br>
are forced to manually build and link a custom sanitized<br>
libc++.<br>
This is a huge PITA and I would like to improve the situation,<br>
not just for MSAN but all sanitizers. I'm working on a proposal<br>
to change this. The basis of my proposal is:<br>
<br>
Clang should install/provide multiple sanitized versions of<br>
Libc++ and a mechanism to easily link them, as if they were a<br>
Compiler-RT runtime.<br>
<br>
The goal of this proposal is:<br>
<br>
(1) Greatly reduce the number of false positives caused by<br>
using<br>
an un-sanitized STL. (2) Allow sanitizers to catch user bugs<br>
that<br>
occur within the STL library, not just its headers.<br>
<br>
The basic steps I would like to take to achieve this are:<br>
<br>
(1) Teach the compiler-rt CMake how to build and install each<br>
sanitized libc++ version along side its other runtimes. (2) Add<br>
options to the Clang driver to support linking/using these<br>
libraries.<br>
<br>
I think this proposal is likely to be contentious, so I would<br>
like to focus on the details it. Once I have some feedback on<br>
these details I'll put together a formal proposal, including a<br>
plan for implementing it. The details I would like input on<br>
are:<br>
<br>
(A) What kind and how many sanitized versions of libc++ should<br>
we provide?<br>
<br>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>--------------------- <br>
<br>
<br>
<br>
</blockquote></blockquote></blockquote>
I think the minimum set would be Address (which includes Leak),<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Memory (With origin tracking?), Thread, and Undefined. Once we<br>
get into combinations of sanitizers things get more<br>
complicated.<br>
What other sanitizer combinations should we provide?<br>
<br>
(B) How should we handle UBSAN?<br>
------------------------------<wbr>---------------------<br>
<br>
UBSAN is really just a collection of sanitizers and providing<br>
sanitized versions of libc++ for every possible configuration<br>
is<br>
out of the question. Instead we should figure out what subset<br>
of<br>
UBSAN checks we want to enable in sanitized libc++ versions. I<br>
suspect we want to disable the following checks.<br>
<br>
* -fsanitize=vptr * -fsanitize=function *<br>
-fsanitize=float-divide-by-zer<wbr>o<br>
<br>
Additionally UBSAN can be combined with every other sanitizer<br>
group (ie Address, Memory, Thread). Do we want to provide a<br>
combination of UBSAN on/off for every group, or can we simply<br>
provide an over-sanitized version with UBSAN on?<br>
<br>
(C) How should the Clang driver expose the sanitized libraries<br>
to the users?<br>
<br>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------------------- <br>
<br>
<br>
<br>
</blockquote></blockquote></blockquote>
I would like to propose the driver option '-fsanitize-stdlib' and<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
'-fsanitize-stdlib=<sanitizer><wbr>'. The first version deduces the<br>
best sanitized version to use, the second allows it to be<br>
explicitly specified.<br>
<br>
A couple of other options are:<br>
<br>
* -fsanitize=foo: Implicitly turn on a sanitized STL. Clang<br>
deduces which version. * -stdlib=libc++-<sanitizer>: Explicitly<br>
turn on and choose a sanitized STL.<br>
<br>
(D) Should sanitized libc++ versions override libc++.so?<br>
<br>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>- <br>
<br>
<br>
<br>
</blockquote></blockquote></blockquote>
For example, what happens when a program links to both a sanitized<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
and non-sanitized libc++ version? Does the sanitized version<br>
replace the non-sanitized version, or should both versions be<br>
loaded into the program?<br>
<br>
Essentially I'm asking if the sanitized versions of libc++<br>
should have the "soname" libc++ so they can replace<br>
non-sanitized<br>
version, or if they should have a different "soname" so the<br>
linker treats them as a separate library.<br>
<br>
I haven't looked into the consequences of either approach in<br>
depth, but any input is appreciated.<br>
</blockquote>
<br>
<br>
In a sense, these are /just/ multilibs, so my inclination would<br>
be<br>
to make all the soname's the same, and just stick them in<br>
appropriately named subfolders relative to their normal<br>
location.<br>
</blockquote>
<br>
<br>
I'm not sure that's true; there's no property of the environment<br>
that<br>
determines which library path you need. As a practical matter, I<br>
can't set $PLATFORM and/or $LIB in my rpath and have ld.so do the<br>
right thing in this context. Moreover, it is really a property of<br>
how<br>
you compiled, so I think using an alternate library name is<br>
natural.<br>
</blockquote>
<br>
<br>
Multilibs solve exactly the problem of "it's a property of how you<br>
compiled". The thing that's subtly different here is that the<br>
usual thing<br>
that people do with multilibs is to provide ABI incompatible<br>
versions of the<br>
same library (which are made incompatible via compiler flags,<br>
-msoft-float,<br>
for example), whereas these libraries just so happen to be ABI<br>
compatible<br>
with their non-instrumented variants.<br>
<br>
I'm not sure I understand what you're saying about $PLATFORM and<br>
$LIB, but I<br>
/think/ it's a red herring: the compiler takes care of adding in<br>
the<br>
multilib suffixes where appropriate, so shouldn't the answer to<br>
"which<br>
library do I stick in the rpath?" include said suffix (when<br>
compiled with<br>
Eric's proposed flag)?<br>
</blockquote>
<br>
What are these suffixes and where are they added?<br>
</blockquote>
<br>
To be clear: the suffixes aren't something that exist yet, but rather<br>
they're something I'm proposing.<br>
<br>
Strawman:<br>
<br>
flag(s) suffix<br>
------- ------<br>
-fsanitize=address /asan<br>
-fsanitize=address,memory /asan/msan<br>
<br>
<br>
Then with `-fsanitize=address`:<br>
<br>
/usr/lib/libc++.so<br>
<br>
becomes:<br>
<br>
/usr/lib/asan/libc++.so<br>
</blockquote>
<br>
This kind of scheme sounds great, but is this something we can implement on our own, or something that requires changes to the dynamic loader (e.g. glibc's ld.so)?<br>
</blockquote>
<br>
Isn't it entirely up to what the user sticks in the rpath of the binaries that they build?<br>
</blockquote></div></div>
It is my understanding that rpath only really helps with executables. If I want to build a dynamic library and sanitize it, without rebuilding my executable, then an rpath won't help.<br>
<br>
Also, it would be awfully nice if -fsanitize=address were the only flag necessary to add to the build to make everything work. Requiring users to add another flag isn't particularly kind.<br>
<br>
If we "only" need to provide sanitizers for libc++, and not libc++abi, then I would be more of a fan of providing a different .so name, along with a lot of version tagging. If version tagging isn't used, then having multiple libc++ versions in the same process would cause all sorts of interposition problems. I don't know how widely available version tagging is in practice though. GNU ld has it ( <a href="http://ftp.gnu.org/old-gnu/Manuals/ld-2.9.1/html_node/ld_25.html" rel="noreferrer" target="_blank">http://ftp.gnu.org/old-gnu/Man<wbr>uals/ld-2.9.1/html_node/ld_25.<wbr>html</a> ).<div><div class="h5"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
Jon<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
-Hal<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
And with `-fsanitize=memory`, you get:<br>
<br>
/usr/lib/asan/msan/libc++.so<br>
<br>
because an msan'd but not asan'd build of the library was not<br>
supplied<br>
by the vendor (for whatever hypothetical reason). Then the validation<br>
problem of having an exponential number of combinations to test<br>
becomes<br>
the vendor's problem: they can ship as many or as few of the flavors<br>
of<br>
the libraries as they want.<br>
<br>
Here you'd have some notion of "satisfies the constraints the user<br>
asked<br>
for" (which is usually "is ABI compatible with" as far as normal<br>
multilib stuff goes) and another to rank the choices and break ties<br>
when<br>
all else is the same.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Note that right now if I build with -stdlib=libc++ (and libc++ is<br>
part<br>
of llvm checkout), I don't get any RPATH. So the binary is linked<br>
against the libc++.so in the toolchain build directory, but it<br>
would<br>
not find it at runtime without some extra help. This is the price<br>
you<br>
pay for running out of temp location, and we should probably keep<br>
it<br>
like this for sanitizer builds, too, i.e. put the sanitized libc++<br>
in<br>
lib/msan and let the user set their own RPATH.<br>
</blockquote>
<br>
Yeah, that's my inclination also. We could of course provide some<br>
flag<br>
to support querying the compiler for what the sanitizer lib suffix is<br>
(or re-use/hijack the existing one for normal multilibs). That'd<br>
allow<br>
build scripts to append the suffix in a principled way.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
The other part of the problem is how to install sanitized libc++<br>
system-wide and have apps use it. That's where we need the loader<br>
support, and I think it should follow the multilib design as close<br>
as<br>
possible.<br>
</blockquote>
<br>
An idea for this: assuming they're all ABI compatible, stick them in<br>
their suffixed folders as appropriate, but add a symlink from the no<br>
suffix location to whatever one you want to be used system-wide.<br>
<br>
<br>
Jon<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
Jon<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
-Hal<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
Jon<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Conclusion -----------------<br>
<br>
I hope my proposal and questions have made sense. Any and all<br>
input is appreciated. Please let me know if anything needs<br>
clarification.<br>
<br>
/Eric<br>
<br>
<br>
<br>
<br>
<br>
<br>
______________________________<wbr>_________________ cfe-dev mailing<br>
list <a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
<br>
</blockquote>
<br>
-- Jon Roelofs <a href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a> CodeSourcery / Mentor<br>
Embedded ______________________________<wbr>_________________ cfe-dev<br>
mailing list <a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
<br>
</blockquote>
<br>
</blockquote>
<br>
-- <br>
Jon Roelofs<br>
<a href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a><br>
CodeSourcery / Mentor Embedded<br>
</blockquote></blockquote>
<br>
-- <br>
Jon Roelofs<br>
<a href="mailto:jonathan@codesourcery.com" target="_blank">jonathan@codesourcery.com</a><br>
CodeSourcery / Mentor Embedded<br>
<br>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<br>
-- <br></div></div>
Employee of Qualcomm Innovation Center, Inc.<br>
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project<div class="HOEnZb"><div class="h5"><br>
<br>
______________________________<wbr>_________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
</div></div></blockquote></div><br></div>