[Lldb-commits] [PATCH] D62499: Create a generic handler for Xfer packets

Mon May 27 15:13:39 PDT 2019

aadsm created this revision.
aadsm added reviewers: clayborg, xiaobai.
Herald added subscribers: lldb-commits, krytarowski.
Herald added a project: LLDB.
aadsm edited the summary of this revision.
aadsm edited the summary of this revision.

This is the first of a few patches I have to improve the performance of dynamic module loading on Android.

In this first diff I'll describe the context of my main motivation and will then link to it in the other diffs to avoid repeating myself.

Motivation
----------

I have a few scenarios where opening a specific feature on an Android app takes around 40s when lldb is attached to it. The reason for that is because 40 modules are dynamicly loaded at that point in time and each one of them is taking ~1s.

The problem
-----------

To learn about new modules we have a breakpoint on a linker function that is called twice whenever a module is loaded. One time just before it's loaded (so lldb can check which modules are loaded) and another right after it's loaded (so lldb can check again which ones are loaded and calculate the diference).
It's figuring out which modules are loaded that is taking quite some time. This is currently done by traversing the linked list of loaded shared libraries that the linker maintains in memory. Each item in the linked list requires its own `x` packet sent to the gdb server (this is android so the network also plays a part). In my scenario there are 400+ loaded libraries and even though we read 0x800 worth of bytes at a time we still make ~180 requests that end up taking 150-200ms.
We also do this twice, once before the module is loaded (state = eAdd) and another right after (state = eConsistent) which easly adds up to ~400ms per module.

A solution
----------

**Implement `xfer:libraries-svr4` in lldb-server:**
I noticed in the code that loads the new modules that it had support for the `xfer:libraries-svr4` packet (added ~4 years ago to support the ds2 debug server) but we didn't support it in lldb-server. This single packet returns an xml list of all the loaded modules by the process. The advantage is that there's no more need to make 180 requests to read the linked list. Additionally this new requests takes around 10ms.

**More efficient usage of the `xfer:libraries-svr4` packet in lldb:**
When `xfer:libraries-svr4` is available the Process class has a `LoadModules` function that requests this packet and then loads or unloads modules based on the current list of loaded modules by the process.
This is the function that is used by the DYLDRendezvous class to get the list of loaded modules before and after the module is loaded. However, this is really not needed since the LoadModules function already loaded or unloaded the modules accordingly. I changed this strategy to call LoadModules only once (after the process has loaded the module).

**Bugs**
I found a few issues in lldb while implementing this and have submitted independent patches for them.

I tried to devide this into multiple logical patches to make it easier to review and discuss.

Tests
-----

I wanted to put these set of diffs up before having all the tests up and running to start having them reviewed from a techical point of view. I'm also having some trouble making the tests running on linux so I need more time to make that happen.

This diff
=========

The `xfer` packages follow the same protocol, they are requested with `xfer:<object>:<read|write>:<annex>:<offset,length>` and a return that starts with `l` or `m` depending if the offset and length covers the entire data or not. Before implementing the `xfer:libraries-svr4` I refactored the `xfer:auxv` to generically handle xfer packets so we can easly add new ones.

The overall structure of the function ends up being:

- Parse the packet into its components: object, offset etc.
- Depending on the object do its own logic to generate the data.
- Return the data based on its size, the requested offset and length.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D62499

Files:
  lldb/include/lldb/Utility/StringExtractor.h
  lldb/include/lldb/Utility/StringExtractorGDBRemote.h
  lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp
  lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.h
  lldb/source/Utility/StringExtractor.cpp
  lldb/source/Utility/StringExtractorGDBRemote.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D62499.201585.patch
Type: text/x-patch
Size: 11196 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20190527/c18846df/attachment-0001.bin>