[PATCH] D148754: [LLD][RFC] Deduplicate type units with local ThinLTO

[Alexander Yermolovich via Phabricator via llvm-commits]

ayermolo created this revision.
Herald added subscribers: hoy, modimo, wenlei, arichardson, inglorion, emaste.
Herald added a reviewer: MaskRay.
Herald added a project: All.
ayermolo requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

I ran into an issue where LLD wasn't de-duplicating type units (TU) with local
ThinLTO. A case is where one TU came from bitcode file build with local ThinLTO,
and another TU came from pre-build ELF object.
With distributed ThinLTO everything was de-duplicated as expected since inputs to
final link are just ELF files. This is a variation of if both type units come
from ThinLTO.

I took a look at https://reviews.llvm.org/D80765, which seems to go nowhere and
primarily just dealt with type units that come from ThinLTO. Due to it's
implementation it wouldn't work for mix mode.

Here I propose a more localized fix for this that just deals with type units. It
deals with a case where all type units come from ThinlTO, and with mix mode
case, and makes it so that behavior is the same between regular and distributed
ThinLTO.

P.S. I will add tests after it's clear that this isn't a dead end.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D148754

Files:
  lld/ELF/InputFiles.cpp


Index: lld/ELF/InputFiles.cpp
===================================================================
--- lld/ELF/InputFiles.cpp
+++ lld/ELF/InputFiles.cpp
@@ -30,6 +30,8 @@
 #include "llvm/Support/RISCVAttributeParser.h"
 #include "llvm/Support/TarWriter.h"
 #include "llvm/Support/raw_ostream.h"
+#include <cstdint>
+#include <utility>
 
 using namespace llvm;
 using namespace llvm::ELF;
@@ -518,6 +520,30 @@
       this);
 }
 
+
+/// For DWARF4 the Type Units are in their own seperate .debug_types section.
+/// For DWARF5 they are part of the .debug_info section.
+/// Returns true if link group contains either of those sections.
+template <class ELFT, class Elf_Word>
+static bool isDebugGroupSection(ObjFile<ELFT> &objFile,
+                                ArrayRef<Elf_Word> &ndxArray,
+                                StringRef &shstrtab) {
+  object::ELFFile<ELFT> obj = objFile.getObj();
+  for (uint32_t ndx : ndxArray) {
+    Expected<const typename ELFT::Shdr *> grpSection = obj.getSection(ndx);
+    if (!grpSection) {
+      warn(toString(&objFile) + " : " +
+           toString(std::move(grpSection.takeError())));
+      break;
+    }
+    StringRef name = check(obj.getSectionName(**grpSection, shstrtab));
+    if ((name.find("debug_types") != StringRef::npos) ||
+        (name.find("debug_info") != StringRef::npos))
+      return true;
+  }
+  return false;
+}
+
 template <class ELFT> void ObjFile<ELFT>::parse(bool ignoreComdats) {
   object::ELFFile<ELFT> obj = this->getObj();
   // Read a section table. justSymbols is usually false.
@@ -583,18 +609,20 @@
 
     if (sec.sh_type != SHT_GROUP)
       continue;
+    StringRef name = check(obj.getSectionName(sec, shstrtab));
     StringRef signature = getShtGroupSignature(objSections, sec);
     ArrayRef<Elf_Word> entries =
         CHECK(obj.template getSectionContentsAsArray<Elf_Word>(sec), this);
     if (entries.empty())
       fatal(toString(this) + ": empty SHT_GROUP");
-
     Elf_Word flag = entries[0];
     if (flag && flag != GRP_COMDAT)
       fatal(toString(this) + ": unsupported SHT_GROUP format");
 
+    ArrayRef<Elf_Word> ndxArray = entries.slice(1);
+    bool isDebugSection = isDebugGroupSection<ELFT, Elf_Word>(*this, ndxArray, shstrtab);
     bool keepGroup =
-        (flag & GRP_COMDAT) == 0 || ignoreComdats ||
+        (flag & GRP_COMDAT) == 0 || (ignoreComdats && !isDebugSection) ||
         symtab.comdatGroups.try_emplace(CachedHashStringRef(signature), this)
             .second;
     if (keepGroup) {
@@ -690,6 +718,8 @@
 template <class ELFT>
 void ObjFile<ELFT>::initializeSections(bool ignoreComdats,
                                        const llvm::object::ELFFile<ELFT> &obj) {
+  static std::mutex io_mutex;
+  std::lock_guard<std::mutex> lk(io_mutex);
   ArrayRef<Elf_Shdr> objSections = getELFShdrs<ELFT>();
   StringRef shstrtab = CHECK(obj.getSectionStringTable(objSections), this);
   uint64_t size = objSections.size();


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D148754.515120.patch
Type: text/x-patch
Size: 2949 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230419/11d3117b/attachment.bin>