[PATCH] D121531: [ELF] Implement --build-id={md5, sha1} with truncated BLAKE3

[Fangrui Song via Phabricator via llvm-commits]

MaskRay created this revision.
MaskRay added reviewers: ikudrin, peter.smith.
Herald added subscribers: dexonsmith, kristof.beyls, arichardson, emaste.
Herald added a project: All.
MaskRay requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

--build-id was introduced as "approximation of true uniqueness across all
binaries that might be used by overlapping sets of people". It does not require
the above mentioned resistance. In practice, people just use --build-id=md5
for 16-byte build ID and --build-id=sha1 for 20-byte build ID.

BLAKE3 has 256-bit key length, which provides 128-bit security against
(second-)preimage, collision, and differentiability attacks. Its portable
implement is fast. It additionally provides Arm Neon/AVX2/AVX-512. Just
implement --build-id={md5,sha1} with truncated BLAKE3.

Linking clang 14 RelWithDebInfo with 8 threads on a Skylake CPU:

- 1.13x as fast with --build-id=md5
- 1.15x as fast with --build-id=sha1


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D121531

Files:
  lld/ELF/Writer.cpp
  lld/test/ELF/build-id.s
  lld/test/ELF/partition-notes.s
  llvm/include/llvm/Support/BLAKE3.h


Index: llvm/include/llvm/Support/BLAKE3.h
===================================================================
--- llvm/include/llvm/Support/BLAKE3.h
+++ llvm/include/llvm/Support/BLAKE3.h
@@ -75,7 +75,7 @@
   static BLAKE3Result<NumBytes> hash(ArrayRef<uint8_t> Data) {
     BLAKE3 Hasher;
     Hasher.update(Data);
-    return Hasher.final();
+    return Hasher.final<NumBytes>();
   }
 
 private:
Index: lld/test/ELF/partition-notes.s
===================================================================
--- lld/test/ELF/partition-notes.s
+++ lld/test/ELF/partition-notes.s
@@ -37,7 +37,7 @@
 // CHECK-NEXT:       Owner: GNU
 // CHECK-NEXT:       Data size:
 // CHECK-NEXT:       Type: NT_GNU_BUILD_ID (unique build ID bitstring)
-// CHECK-NEXT:       Build ID: bb5542bd74252653e286044980d602874d237ae0
+// CHECK-NEXT:       Build ID: ab81108a3d85b729980356331fddc2bfc4c10177{{$}}
 // CHECK-NEXT:     }
 // CHECK-NEXT:   }
 // CHECK-NEXT: ]
Index: lld/test/ELF/build-id.s
===================================================================
--- lld/test/ELF/build-id.s
+++ lld/test/ELF/build-id.s
@@ -69,11 +69,11 @@
 
 # MD5:      Contents of section .note.gnu.build-id:
 # MD5-NEXT: 04000000 10000000 03000000 474e5500  ............GNU.
-# MD5-NEXT: 7b00fd9e 054ceb4b 06f64d0e 482cb476
+# MD5-NEXT: dbf0bc13 b3ff11e9 fde6e17c 0304983c
 
 # SHA1:      Contents of section .note.gnu.build-id:
 # SHA1-NEXT: 04000000 14000000 03000000 474e5500  ............GNU.
-# SHA1-NEXT: 221a99da dd1d2bf3 05e48a91 dde8a0cb
+# SHA1-NEXT: 1215775f d3b60050 70afd970 e8a10972
 
 # UUID:      Contents of section .note.gnu.build-id:
 # UUID-NEXT: 04000000 10000000 03000000 474e5500  ............GNU.
@@ -89,11 +89,11 @@
 
 # SEPARATE:      Hex dump of section '.note.gnu.build-id':
 # SEPARATE-NEXT: 0x00200198 04000000 14000000 03000000 474e5500
-# SEPARATE-NEXT: 0x002001a8 96820adf d90d5470 0a0c32ff a88c4017
+# SEPARATE-NEXT: 0x002001a8 5cd067a4 2631c0fd 42029037 4b8e0938
 
 # RUN: ld.lld --build-id=sha1 --no-rosegment %t -o %t2
 # RUN: llvm-readelf -x .note.gnu.build-id %t2 | FileCheck --check-prefix=NORO %s
 
 # NORO:      Hex dump of section '.note.gnu.build-id':
 # NORO-NEXT: 0x00200160 04000000 14000000 03000000 474e5500
-# NORO-NEXT: 0x00200170 cf6d7b3a 0b3297c3 5b47c079 ce048349
+# NORO-NEXT: 0x00200170 a328cc99 45bfc3fc a9fc8615 37102f9d
Index: lld/ELF/Writer.cpp
===================================================================
--- lld/ELF/Writer.cpp
+++ lld/ELF/Writer.cpp
@@ -25,6 +25,7 @@
 #include "lld/Common/Filesystem.h"
 #include "lld/Common/Strings.h"
 #include "llvm/ADT/StringMap.h"
+#include "llvm/Support/BLAKE3.h"
 #include "llvm/Support/MD5.h"
 #include "llvm/Support/Parallel.h"
 #include "llvm/Support/RandomNumberGenerator.h"
@@ -2933,12 +2934,12 @@
     break;
   case BuildIdKind::Md5:
     computeHash(output, input, [&](uint8_t *dest, ArrayRef<uint8_t> arr) {
-      memcpy(dest, MD5::hash(arr).data(), hashSize);
+      memcpy(dest, BLAKE3::hash<16>(arr).data(), hashSize);
     });
     break;
   case BuildIdKind::Sha1:
     computeHash(output, input, [&](uint8_t *dest, ArrayRef<uint8_t> arr) {
-      memcpy(dest, SHA1::hash(arr).data(), hashSize);
+      memcpy(dest, BLAKE3::hash<20>(arr).data(), hashSize);
     });
     break;
   case BuildIdKind::Uuid:


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D121531.414865.patch
Type: text/x-patch
Size: 3294 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20220312/1964b18c/attachment.bin>