[llvm-dev] Memory sanitizer porting

[vit9696 via llvm-dev]

Hello,

I am currently porting memory sanitizer to a custom platform, and discovered some strange things in the existing implementation.

1. clang/llvm currently hardcode the list of supported platforms and disallow the use of a standalone msan implementation.
I suppose the solution here is to submit a patch similar to https://reviews.llvm.org/D18865 <https://reviews.llvm.org/D18865>, which will provide the necessary arguments to configure the layout.
I have it ready here. Will this approach be fine for the llvm dev team, and may I post it for the review then?

2. There exists a concept of -fsanitize-blacklist argument, which is supposed to exclude source locations from being instrumented by the sanitizers.
Normally this works without issues, but I discovered that you cannot compile compiler-rt msan implementation with -fsanitize=memory even if the whole location is blacklisted (that much I can confirm from the invocations of CodeGenModule::isInSanitizerBlacklist function).
What happens is that for some reason memory sanitizer:
— still tries to partially instrument the blacklisted code;
— does not check whether its global memory storage variables are already defined.
The second issue, present in MemorySanitizer::initializeCallbacks, adds a second copy of storage global variables when compiling msan.cc (e.g. __emutls_v.__msan_retval_tls.63, __emutls_v.__msan_param_tls.65), and this results in an undefined reference during the linkage.

Here the question is what was initially intended to be done. I know that compiling blacklisted asan runtime code with -fsanitize=address is just fine, and this is what Apple actually does in XNU KASAN implementation.
I kind of expected it to be the right way to do for msan as well, is this just a bug? If it is not, should I compile msan runtime without -fsanitize=memory in this case, and in fact asan runtime too?

3. Other than that, I see that memory sanitizer is currently implemented only for 64-bit platforms. While I am aware of the issues behind requiring a lot of memory to use msan, are there any other issues for not supporting 32-bit?

Best regards,
Vit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180222/63573031/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180222/63573031/attachment.sig>