[compiler-rt] r275207 - [asan] Fix interception unittest on Windows64.

[Etienne Bergeron via llvm-commits]

Author: etienneb
Date: Tue Jul 12 14:39:07 2016
New Revision: 275207

URL: http://llvm.org/viewvc/llvm-project?rev=275207&view=rev
Log:
[asan] Fix interception unittest on Windows64.

mov edi,edi is _not_ NOP in 64-bit, use 66,90h instead.
This bug was causing interception unittest to crash on 
Windows64 (windows 8 and windows 10).

Credits to etienneb for finding the root cause.

Patch by: Wei Wang
Differential Revision: http://reviews.llvm.org/D22274


Modified:
    compiler-rt/trunk/lib/interception/interception_win.cc
    compiler-rt/trunk/lib/interception/tests/interception_win_test.cc

Modified: compiler-rt/trunk/lib/interception/interception_win.cc
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/interception/interception_win.cc?rev=275207&r1=275206&r2=275207&view=diff
==============================================================================
--- compiler-rt/trunk/lib/interception/interception_win.cc (original)
+++ compiler-rt/trunk/lib/interception/interception_win.cc Tue Jul 12 14:39:07 2016
@@ -461,6 +461,7 @@ static size_t GetInstructionSize(uptr ad
     case 0x5541:  // push r13
     case 0x5641:  // push r14
     case 0x5741:  // push r15
+    case 0x9066:  // Two-byte NOP
       return 2;
   }
 

Modified: compiler-rt/trunk/lib/interception/tests/interception_win_test.cc
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/interception/tests/interception_win_test.cc?rev=275207&r1=275206&r2=275207&view=diff
==============================================================================
--- compiler-rt/trunk/lib/interception/tests/interception_win_test.cc (original)
+++ compiler-rt/trunk/lib/interception/tests/interception_win_test.cc Tue Jul 12 14:39:07 2016
@@ -234,8 +234,18 @@ static void LoadActiveCode(
 
   // Add the detour instruction (i.e. mov edi, edi)
   if (prefix_kind == FunctionPrefixDetour) {
+#if SANITIZER_WINDOWS64
+    // Note that "mov edi,edi" is NOP in 32-bit only, in 64-bit it clears
+    // higher bits of RDI.
+    // Use 66,90H as NOP for Windows64.
+    ActiveCode[position++] = 0x66;
+    ActiveCode[position++] = 0x90;
+#else
+    // mov edi,edi.
     ActiveCode[position++] = 0x8B;
     ActiveCode[position++] = 0xFF;
+#endif
+
   }
 
   // Copy the function body.