[PATCH] D73502: [BinaryFormat] Limit COFF file detection with unknown machine type

[Sam Clegg via Phabricator via llvm-commits]

sbc100 created this revision.
Herald added subscribers: llvm-commits, aheejin, hiraditya.
Herald added a project: LLVM.
sbc100 added a subscriber: alexcrichton.
sbc100 added a reviewer: Bigcheese.
Herald added a subscriber: dexonsmith.

When the machine type is zero (first two bytes) also check that we
have a non-zero number of sections (second two bytes).  This avoids
mistaking any file that has two leaving zeros as a COFF file.

Fixes PR44683


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D73502

Files:
  llvm/lib/BinaryFormat/Magic.cpp
  llvm/unittests/BinaryFormat/TestFileMagic.cpp


Index: llvm/unittests/BinaryFormat/TestFileMagic.cpp
===================================================================
--- llvm/unittests/BinaryFormat/TestFileMagic.cpp
+++ llvm/unittests/BinaryFormat/TestFileMagic.cpp
@@ -84,6 +84,7 @@
                    "DS\x00\x00\x00";
 const char tapi_file[] = "--- !tapi-tbd-v1\n";
 const char tapi_file_tbd_v1[] = "---\narchs: [";
+const char all_zeros[] = "\x00\x00\x00\x00";
 
 TEST_F(MagicTest, Magic) {
   struct type {
@@ -119,6 +120,8 @@
       DEFINE(tapi_file),
       {"tapi_file_tbd_v1", tapi_file_tbd_v1, sizeof(tapi_file_tbd_v1),
        file_magic::tapi_file},
+      // A file containing all zeros should not be recognized as a valid object
+      {"all_zeros", all_zeros, sizeof(all_zeros), file_magic::unknown},
 #undef DEFINE
   };
 
Index: llvm/lib/BinaryFormat/Magic.cpp
===================================================================
--- llvm/lib/BinaryFormat/Magic.cpp
+++ llvm/lib/BinaryFormat/Magic.cpp
@@ -55,8 +55,15 @@
         memcmp(Magic.data(), COFF::WinResMagic, sizeof(COFF::WinResMagic)) == 0)
       return file_magic::windows_resource;
     // 0x0000 = COFF unknown machine type
-    if (Magic[1] == 0)
-      return file_magic::coff_object;
+    if (Magic[1] == 0) {
+      // We don't want to mis-identify a file that simply have 2 leading null
+      // bytes as a COFF object.  One example is the rust.metadata.bin which
+      // the rust toolchain embedds in ar archives. This file starts with 4
+      // leading null bytes.  To work around check the following two bytes which
+      // represet the number of COFF sections.
+      if (Magic[2] != 0 || Magic[3] != 0)
+        return file_magic::coff_object;
+    }
     if (startswith(Magic, "\0asm"))
       return file_magic::wasm_object;
     break;


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D73502.240664.patch
Type: text/x-patch
Size: 1794 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20200127/667caaa0/attachment.bin>