[lld] 2b9b965 - [LLD][COFF] Reduce the maximum size of the GHASH table

[Alexandre Ganea via llvm-commits]

Author: Alexandre Ganea
Date: 2021-05-28T09:46:17-04:00
New Revision: 2b9b9652ce27467283ae90a767af57d838a2465b

URL: https://github.com/llvm/llvm-project/commit/2b9b9652ce27467283ae90a767af57d838a2465b
DIFF: https://github.com/llvm/llvm-project/commit/2b9b9652ce27467283ae90a767af57d838a2465b.diff

LOG: [LLD][COFF] Reduce the maximum size of the GHASH table

Before this patch, the maximum size of the GHASH table was 2^31 buckets. However we were storing the bucket index into a TypeIndex which has an input limit of (2^31)-4095 indices, see this link. Any value above that limit will improperly set the TypeIndex's high bit, which is interpreted as DecoratedItemIdMask. This used to cause bad indices on extraction when calling TypeIndex::toArrayIndex().

Differential Revision: https://reviews.llvm.org/D103297

Added: 
    

Modified: 
    lld/COFF/DebugTypes.cpp

Removed: 
    


################################################################################
diff  --git a/lld/COFF/DebugTypes.cpp b/lld/COFF/DebugTypes.cpp
index bda9b7728d952..5e8bdef5945dc 100644
--- a/lld/COFF/DebugTypes.cpp
+++ b/lld/COFF/DebugTypes.cpp
@@ -1071,7 +1071,8 @@ void TypeMerger::mergeTypesWithGHash() {
 
   // Cap the table size so that we can use 32-bit cell indices. Type indices are
   // also 32-bit, so this is an inherent PDB file format limit anyway.
-  tableSize = std::min(size_t(INT32_MAX), tableSize);
+  tableSize =
+      std::min(size_t(INT32_MAX) - TypeIndex::FirstNonSimpleIndex, tableSize);
   ghashState.table.init(static_cast<uint32_t>(tableSize));
 
   // Insert ghashes in parallel. During concurrent insertion, we cannot observe