[Lldb-commits] [lldb] [lldb][Darwin] Fetch detailed binary info in chunks (PR #190720)
Jason Molenda via lldb-commits
lldb-commits at lists.llvm.org
Thu Apr 9 18:17:10 PDT 2026
https://github.com/jasonmolenda updated https://github.com/llvm/llvm-project/pull/190720
>From 3c11b2ce7bb20f14560b7b8643a469b2a0e1b3d2 Mon Sep 17 00:00:00 2001
From: Jason Molenda <jmolenda at apple.com>
Date: Mon, 6 Apr 2026 18:13:47 -0700
Subject: [PATCH 1/2] [lldb][Darwin] Fetch detailed binary info in chunks
When binaries have been loaded into a process on Darwin, lldb sends
a jGetLoadedDynamicLibrariesInfos packet to get the filepath, uuid,
load address, and detailed information from the mach header/load
commands. For a large UI app, the number of binaries that can be
loaded (through various dependencies) can exceed a thousand these
days, and requesting detailed information on all of those can
result in debugserver allocating too much memory when running in
constrained environments, and being killed.
In 2023 I laid the groundwork to fetch detailed information in
chunks, instead of one large request. The main challenge with this
is when we first attach to a process that is running, we send a
"tell me about all binaries loaded", and that prevents lldb from
chunking the reply; the packet design for jGetLoadedDynamicLibrariesInfos
assumes the entire reply is sent in one packet, instead of the
typical gdb remote serial protocol trick of a response with partial
data starting with 'm' and a response with a complete reply starting
with 'l'. The 2023 change is to add a new key to this packet,
`report_load_commands` and when that is set to `false`, only the
load address of the binaries is reported.
lldb then uses the array of load addresses of all the binaries to
fetch detailed information about them in smaller groupings.
This PR implements the lldb side of that work.
Process::GetLoadedDynamicLibrariesInfos now takes a `bool
include_mh_and_load_commands`, ProcessGDBRemote sends that
as an argument in the jGetLoadedDynamicLibrariesInfos packet.
DynamicLoaderMacOS::DoInitialImageFetch is changed to only get
the load addresses on initial attach. If the reply includes
the full binary information (not just load addresses) -- when
talking to an old debugserver -- we will use that information
instead of re-fetching it. On a newer debugserver that only
sent the load addresses, we'll send this list of addresses
to the standard method we use when dyld has told us to load
binaries at addresses already.
DynamicLoaderMacOS::AddBinaries, which takes a list of addresses
and fetches detailed information about them, is updated to request
only 600 binaries at a time. A typical UI app will be in the
700-1000 binary range these days, so this will turn one large fetch
into two, in most cases. There are some system UI processes that
have many dependencies that could require three fetches. I picked
this number so most debug sessions will be handled by two requests.
In debugserver MachProcess::FormatDynamicLibrariesIntoJSON, I removed
the obsolete-for-three-years-now `mod_date` field. I was sending
back the binary filepaths for this "don't send the detailed information"
version of the packet - I don't need that, and it just increases the
size, so I stopped sending filepaths in this mode.
I also added a new field for when we ARE sending detailed information,
`sizeof_mh_and_loadcmds`. I don't use this in lldb yet, but when
we are told about a binary and need to read it from memory today,
we have an initial read to get the mach header, which tells us the
size of the load commands. Then we have a second read of the mach
header plus load commands, before we can start binary processing
in earnest. This is an extra read packet and very unnecessary,
given that debugserver knows how large the mach header + load
commands are. So I'm returning it here, and at some point I'll
find a way to pipe that into a new memory object file creation
method in lldb. It's one of those "I should really find a way to
remove that extra read some day" cleanups, and while I was in
this area, I'd add this first piece of that.
I don't have a test for this. I've been thinking about an API test
that creates 700 dylibs with empty functions in each, runs it, and
confirms all of the dylibs were loaded. I'd have to grab a packet
log to be completely sure we didn't read the full binary list in
one go. But I worry that compiling and linking even 700 do-nothing
dylibs might be too much. Maybe I should add a setting in
DynamicLoaderMacOS::AddBinaries to reduce the maximum number of
binaries that can be read at once, and have a small nubmer of dylibs.
When by-hand testing this, I had a maximum of 5 binaries being
queried in one packet.
rdar://109428337
---
lldb/include/lldb/Target/Process.h | 35 +++++-
.../MacOSX-DYLD/DynamicLoaderMacOS.cpp | 106 +++++++++++++-----
.../Process/gdb-remote/ProcessGDBRemote.cpp | 5 +-
.../Process/gdb-remote/ProcessGDBRemote.h | 3 +-
.../Process/scripted/ScriptedProcess.cpp | 3 +-
.../Process/scripted/ScriptedProcess.h | 4 +-
.../debugserver/source/MacOSX/MachProcess.mm | 14 ++-
7 files changed, 124 insertions(+), 46 deletions(-)
diff --git a/lldb/include/lldb/Target/Process.h b/lldb/include/lldb/Target/Process.h
index 7fe9aa3f5ab1b..cc73f249577a4 100644
--- a/lldb/include/lldb/Target/Process.h
+++ b/lldb/include/lldb/Target/Process.h
@@ -1332,15 +1332,40 @@ class Process : public std::enable_shared_from_this<Process>,
return StructuredData::ObjectSP();
}
- // On macOS 10.12, tvOS 10, iOS 10, watchOS 3 and newer, debugserver can
- // return the full list of loaded shared libraries without needing any input.
+ /// Retrieve a StructuredData dictionary about all of the binaries
+ /// loaded in the process at this time.
+ /// A Darwin target specific behavior, only supported by debugserver,
+ /// response will include load address, filepath, uuid, and may also
+ /// include the fully parsed mach header and load commands.
+ ///
+ /// \param [in] include_mh_and_load_commands
+ /// Whether the remote stub should include the full details of
+ /// the mach header and load commands in its reply. This may
+ /// cause the packet size to be quite large, so to avoid memory
+ /// pressure in the remote stub, caller may choose not to fetch
+ /// these for all binaries.
+ ///
+ /// \return
+ /// A StructuredData object with the information that could be
+ /// retrieved..
virtual lldb_private::StructuredData::ObjectSP
- GetLoadedDynamicLibrariesInfos() {
+ GetLoadedDynamicLibrariesInfos(bool include_mh_and_load_commands) {
return StructuredData::ObjectSP();
}
- // On macOS 10.12, tvOS 10, iOS 10, watchOS 3 and newer, debugserver can
- // return information about binaries given their load addresses.
+ /// Retrieve a StructuredData dictionary about the binaries at
+ /// the provided load addresses.
+ /// A Darwin target specific behavior, only supported by debugserver,
+ /// response will include load address, filepath, uuid, fully parsed
+ /// mach header and load commands.
+ ///
+ /// \param [in] load_addresses
+ /// The virtual address of the start of binaries to fetch
+ /// information.
+ ///
+ /// \return
+ /// A StructuredData object with the information that could be
+ /// retrieved..
virtual lldb_private::StructuredData::ObjectSP GetLoadedDynamicLibrariesInfos(
const std::vector<lldb::addr_t> &load_addresses) {
return StructuredData::ObjectSP();
diff --git a/lldb/source/Plugins/DynamicLoader/MacOSX-DYLD/DynamicLoaderMacOS.cpp b/lldb/source/Plugins/DynamicLoader/MacOSX-DYLD/DynamicLoaderMacOS.cpp
index feafa82c6eff4..21d7b9b4201a6 100644
--- a/lldb/source/Plugins/DynamicLoader/MacOSX-DYLD/DynamicLoaderMacOS.cpp
+++ b/lldb/source/Plugins/DynamicLoader/MacOSX-DYLD/DynamicLoaderMacOS.cpp
@@ -189,10 +189,6 @@ void DynamicLoaderMacOS::ClearNotificationBreakpoint() {
}
}
-// Try and figure out where dyld is by first asking the Process if it knows
-// (which currently calls down in the lldb::Process to get the DYLD info
-// (available on SnowLeopard only). If that fails, then check in the default
-// addresses.
void DynamicLoaderMacOS::DoInitialImageFetch() {
Log *log = GetLog(LLDBLog::DynamicLoader);
@@ -203,7 +199,8 @@ void DynamicLoaderMacOS::DoInitialImageFetch() {
UnloadAllImages();
StructuredData::ObjectSP all_image_info_json_sp(
- m_process->GetLoadedDynamicLibrariesInfos());
+ m_process->GetLoadedDynamicLibrariesInfos(
+ /*include_mh_and_load_commands=*/false));
ImageInfo::collection image_infos;
if (all_image_info_json_sp.get() &&
all_image_info_json_sp->GetAsDictionary() &&
@@ -211,14 +208,44 @@ void DynamicLoaderMacOS::DoInitialImageFetch() {
all_image_info_json_sp->GetAsDictionary()
->GetValueForKey("images")
->GetAsArray()) {
- if (JSONImageInformationIntoImageInfo(all_image_info_json_sp,
- image_infos)) {
- LLDB_LOGF(log, "Initial module fetch: Adding %" PRId64 " modules.\n",
- (uint64_t)image_infos.size());
-
- auto images = PreloadModulesFromImageInfos(image_infos);
- UpdateSpecialBinariesFromPreloadedModules(images);
- AddModulesUsingPreloadedModules(images);
+
+ // Older debugserver (pre-2024-ish) will not recognize the
+ // include_mh_and_load_commands==false option above, and
+ // will return the full binary information including mach
+ // header and segments/load commands. The response includes
+ // the full information on all binaries.
+ StructuredData::Array *images = all_image_info_json_sp->GetAsDictionary()
+ ->GetValueForKey("images")
+ ->GetAsArray();
+ if (images->GetSize() > 0 &&
+ images->GetItemAtIndex(0)->GetAsDictionary()->HasKey("mach_header")) {
+ if (JSONImageInformationIntoImageInfo(all_image_info_json_sp,
+ image_infos)) {
+ LLDB_LOGF(log, "Initial module fetch: Adding %" PRId64 " modules.\n",
+ (uint64_t)image_infos.size());
+
+ auto images = PreloadModulesFromImageInfos(image_infos);
+ UpdateSpecialBinariesFromPreloadedModules(images);
+ AddModulesUsingPreloadedModules(images);
+ }
+ } else {
+ // This is a newer debugserver which only replied with
+ // `load_address` for all binaries loaded in the process.
+ // We can request detailed information in smaller chunks,
+ // instead of one gigantic packet.
+ size_t image_count = images->GetSize();
+ std::vector<addr_t> load_addresses;
+ for (size_t i = 0; i < image_count; i++) {
+ StructuredData::Dictionary *image =
+ images->GetItemAtIndex(i)->GetAsDictionary();
+ if (image->HasKey("load_address")) {
+ addr_t val = image->GetValueForKey("load_address")
+ ->GetUnsignedIntegerValue(LLDB_INVALID_ADDRESS);
+ if (val != LLDB_INVALID_ADDRESS)
+ load_addresses.push_back(val);
+ }
+ }
+ AddBinaries(load_addresses);
}
}
@@ -413,26 +440,43 @@ void DynamicLoaderMacOS::AddBinaries(
Log *log = GetLog(LLDBLog::DynamicLoader);
ImageInfo::collection image_infos;
- LLDB_LOGF(log, "Adding %" PRId64 " modules.",
- (uint64_t)load_addresses.size());
- StructuredData::ObjectSP binaries_info_sp =
- m_process->GetLoadedDynamicLibrariesInfos(load_addresses);
- if (binaries_info_sp.get() && binaries_info_sp->GetAsDictionary() &&
- binaries_info_sp->GetAsDictionary()->HasKey("images") &&
- binaries_info_sp->GetAsDictionary()
- ->GetValueForKey("images")
- ->GetAsArray() &&
- binaries_info_sp->GetAsDictionary()
- ->GetValueForKey("images")
- ->GetAsArray()
- ->GetSize() == load_addresses.size()) {
- if (JSONImageInformationIntoImageInfo(binaries_info_sp, image_infos)) {
- auto images = PreloadModulesFromImageInfos(image_infos);
- UpdateSpecialBinariesFromPreloadedModules(images);
- AddModulesUsingPreloadedModules(images);
+ const size_t image_fetch_max = 600;
+ std::vector<addr_t> fetch_binaries;
+ size_t fetched = 0;
+ size_t total_image_size = load_addresses.size();
+ fetch_binaries.reserve(std::min(image_fetch_max, total_image_size));
+ while (fetched < total_image_size) {
+ size_t this_fetch_amt =
+ std::min(image_fetch_max, total_image_size - fetched);
+ fetch_binaries.resize(this_fetch_amt);
+ // `addr_t* + num_elem` -- pointer math is addr_t sized.
+ const addr_t *this_chunk_start = load_addresses.data() + fetched;
+ memcpy(fetch_binaries.data(), this_chunk_start,
+ this_fetch_amt * sizeof(addr_t));
+
+ LLDB_LOGF(log, "Adding %" PRId64 " modules.",
+ (uint64_t)fetch_binaries.size());
+ image_infos.clear();
+ StructuredData::ObjectSP binaries_info_sp =
+ m_process->GetLoadedDynamicLibrariesInfos(fetch_binaries);
+ if (binaries_info_sp.get() && binaries_info_sp->GetAsDictionary() &&
+ binaries_info_sp->GetAsDictionary()->HasKey("images") &&
+ binaries_info_sp->GetAsDictionary()
+ ->GetValueForKey("images")
+ ->GetAsArray()) {
+ StructuredData::Array *images = binaries_info_sp->GetAsDictionary()
+ ->GetValueForKey("images")
+ ->GetAsArray();
+ if (images->GetSize() == fetch_binaries.size() &&
+ JSONImageInformationIntoImageInfo(binaries_info_sp, image_infos)) {
+ auto images = PreloadModulesFromImageInfos(image_infos);
+ UpdateSpecialBinariesFromPreloadedModules(images);
+ AddModulesUsingPreloadedModules(images);
+ }
}
- m_dyld_image_infos_stop_id = m_process->GetStopID();
+ fetched += this_fetch_amt;
}
+ m_dyld_image_infos_stop_id = m_process->GetStopID();
}
// Dump the _dyld_all_image_infos members and all current image infos that we
diff --git a/lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp b/lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp
index d74649a48405d..20fcb71004c9e 100644
--- a/lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp
+++ b/lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp
@@ -4245,10 +4245,13 @@ StructuredData::ObjectSP ProcessGDBRemote::GetLoadedDynamicLibrariesInfos(
return GetLoadedDynamicLibrariesInfos_sender(args_dict);
}
-StructuredData::ObjectSP ProcessGDBRemote::GetLoadedDynamicLibrariesInfos() {
+StructuredData::ObjectSP ProcessGDBRemote::GetLoadedDynamicLibrariesInfos(
+ bool include_mh_and_load_commands) {
StructuredData::ObjectSP args_dict(new StructuredData::Dictionary());
args_dict->GetAsDictionary()->AddBooleanItem("fetch_all_solibs", true);
+ args_dict->GetAsDictionary()->AddBooleanItem("report_load_commands",
+ include_mh_and_load_commands);
return GetLoadedDynamicLibrariesInfos_sender(args_dict);
}
diff --git a/lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.h b/lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.h
index 434c4f29201e5..14bcb3e13d121 100644
--- a/lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.h
+++ b/lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.h
@@ -233,7 +233,8 @@ class ProcessGDBRemote : public Process,
ConfigureStructuredData(llvm::StringRef type_name,
const StructuredData::ObjectSP &config_sp) override;
- StructuredData::ObjectSP GetLoadedDynamicLibrariesInfos() override;
+ StructuredData::ObjectSP
+ GetLoadedDynamicLibrariesInfos(bool include_mh_and_load_commands) override;
StructuredData::ObjectSP GetLoadedDynamicLibrariesInfos(
const std::vector<lldb::addr_t> &load_addresses) override;
diff --git a/lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp b/lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp
index b1327964bb69a..0bc76a87f8075 100644
--- a/lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp
+++ b/lldb/source/Plugins/Process/scripted/ScriptedProcess.cpp
@@ -420,7 +420,8 @@ bool ScriptedProcess::GetProcessInfo(ProcessInstanceInfo &info) {
}
lldb_private::StructuredData::ObjectSP
-ScriptedProcess::GetLoadedDynamicLibrariesInfos() {
+ScriptedProcess::GetLoadedDynamicLibrariesInfos(
+ bool include_mh_and_load_commands) {
Status error;
auto error_with_message = [&error](llvm::StringRef message) {
return ScriptedInterface::ErrorWithMessage<bool>(LLVM_PRETTY_FUNCTION,
diff --git a/lldb/source/Plugins/Process/scripted/ScriptedProcess.h b/lldb/source/Plugins/Process/scripted/ScriptedProcess.h
index dad4e6f1e61fe..1ff1c89b68c16 100644
--- a/lldb/source/Plugins/Process/scripted/ScriptedProcess.h
+++ b/lldb/source/Plugins/Process/scripted/ScriptedProcess.h
@@ -85,7 +85,7 @@ class ScriptedProcess : public Process {
bool GetProcessInfo(ProcessInstanceInfo &info) override;
lldb_private::StructuredData::ObjectSP
- GetLoadedDynamicLibrariesInfos() override;
+ GetLoadedDynamicLibrariesInfos(bool include_mh_and_load_commands) override;
lldb_private::StructuredData::DictionarySP GetMetadata() override;
@@ -98,7 +98,7 @@ class ScriptedProcess : public Process {
// dictionary before emitting the private stop event to avoid having the
// module loading happen while the process state is changing.
if (StateIsStoppedState(state, true))
- GetLoadedDynamicLibrariesInfos();
+ GetLoadedDynamicLibrariesInfos(true);
SetPrivateState(state);
}
diff --git a/lldb/tools/debugserver/source/MacOSX/MachProcess.mm b/lldb/tools/debugserver/source/MacOSX/MachProcess.mm
index dce86d479bb7a..4ec84be0f227d 100644
--- a/lldb/tools/debugserver/source/MacOSX/MachProcess.mm
+++ b/lldb/tools/debugserver/source/MacOSX/MachProcess.mm
@@ -969,17 +969,17 @@ static bool mach_header_validity_test(uint32_t magic, uint32_t cputype) {
new JSONGenerator::Dictionary());
image_info_dict_sp->AddIntegerItem("load_address",
image_infos[i].load_address);
- // TODO: lldb currently rejects a response without this, but it
- // is always zero from dyld. It can be removed once we've had time
- // for lldb's that require it to be present are obsolete.
- image_info_dict_sp->AddIntegerItem("mod_date", 0);
- image_info_dict_sp->AddStringItem("pathname", image_infos[i].filename);
+ // If we're not in `report_load_commands` mode, only send back
+ // the mach-o header load addresses; we will fetch the full
+ // binary information later in chunks.
if (!report_load_commands) {
image_infos_array_sp->AddItem(image_info_dict_sp);
continue;
}
+ image_info_dict_sp->AddStringItem("pathname", image_infos[i].filename);
+
uuid_string_t uuidstr;
uuid_unparse_upper(image_infos[i].macho_info.uuid, uuidstr);
image_info_dict_sp->AddStringItem("uuid", uuidstr);
@@ -1006,6 +1006,10 @@ static bool mach_header_validity_test(uint32_t magic, uint32_t cputype) {
"filetype", image_infos[i].macho_info.mach_header.filetype);
mach_header_dict_sp->AddIntegerItem ("flags",
image_infos[i].macho_info.mach_header.flags);
+ mach_header_dict_sp->AddIntegerItem(
+ "sizeof_mh_and_loadcmds",
+ sizeof(mach_header_64) +
+ image_infos[i].macho_info.mach_header.sizeofcmds);
// DynamicLoaderMacOSX doesn't currently need these fields, so
// don't send them.
>From 8482c0b77899a532446a07bf2f05d5c9c14b7a40 Mon Sep 17 00:00:00 2001
From: Jason Molenda <github-mail at molenda.com>
Date: Thu, 9 Apr 2026 18:17:02 -0700
Subject: [PATCH 2/2] Update lldb/include/lldb/Target/Process.h
Co-authored-by: Jonas Devlieghere <jonas at devlieghere.com>
---
lldb/include/lldb/Target/Process.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lldb/include/lldb/Target/Process.h b/lldb/include/lldb/Target/Process.h
index cc73f249577a4..f666147b884be 100644
--- a/lldb/include/lldb/Target/Process.h
+++ b/lldb/include/lldb/Target/Process.h
@@ -1347,7 +1347,7 @@ class Process : public std::enable_shared_from_this<Process>,
///
/// \return
/// A StructuredData object with the information that could be
- /// retrieved..
+ /// retrieved.
virtual lldb_private::StructuredData::ObjectSP
GetLoadedDynamicLibrariesInfos(bool include_mh_and_load_commands) {
return StructuredData::ObjectSP();
More information about the lldb-commits
mailing list