[PATCH] D44225: Fix identification of COFF executable files

[Zachary Turner via Phabricator via llvm-commits]

zturner created this revision.
zturner added reviewers: rnk, compnerd, majnemer.
Herald added a subscriber: hiraditya.

`identify_magic` on COFF executables has been broken for apparently quite some time.  The reason is because we only read the first 32 bytes of the file, but the detection algorithm for COFF requires at *minimum* 64 bytes, but in practice you need to read the entire file because the algorithm reads a 4-byte value at offset 60 and then treats that as an offset, from which it reads additional data.


https://reviews.llvm.org/D44225

Files:
  llvm/lib/BinaryFormat/Magic.cpp


Index: llvm/lib/BinaryFormat/Magic.cpp
===================================================================
--- llvm/lib/BinaryFormat/Magic.cpp
+++ llvm/lib/BinaryFormat/Magic.cpp
@@ -14,6 +14,7 @@
 #include "llvm/BinaryFormat/MachO.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/FileSystem.h"
+#include "llvm/Support/MemoryBuffer.h"
 
 #if !defined(_MSC_VER) && !defined(__MINGW32__)
 #include <unistd.h>
@@ -214,6 +215,22 @@
   if (close(FD) != 0 || Length < 0)
     return std::error_code(errno, std::generic_category());
 
-  Result = identify_magic(StringRef(Buffer, Length));
+  std::unique_ptr<MemoryBuffer> FileBuffer;
+  StringRef BufferStr(Buffer, Length);
+  // If this might be a PE file, we need to read the whole file, because
+  // verification requires seeking to a certain offset, which is only determined
+  // by examining a field in the file.  Since we don't know what that offset
+  // will be we have to assume it could be anywhere, and thus we need to read
+  // the entire file.
+  if (BufferStr.startswith("MZ")) {
+    auto FileOrError = MemoryBuffer::getFile(Path);
+    if (!FileOrError)
+      return FileOrError.getError();
+    FileBuffer = std::move(*FileOrError);
+    BufferStr = FileBuffer->getBuffer();
+  }
+
+  Result = identify_magic(BufferStr);
+
   return std::error_code();
 }


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D44225.137465.patch
Type: text/x-patch
Size: 1332 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180307/04235e6b/attachment.bin>