[Lldb-commits] [PATCH] D153060: lldb: do more than 1 kilobyte at a time to vastly increase binary sync speed

[Russell Greene via Phabricator via lldb-commits]

russelltg created this revision.
Herald added a project: All.
russelltg requested review of this revision.
Herald added a project: LLDB.
Herald added a subscriber: lldb-commits.

https://github.com/llvm/llvm-project/issues/62750

I setup a simple test with a large .so (~100MiB) that was only present on the target machine
but not present on the local machine, and ran a lldb server on the target and connectd to it.

LLDB properly downloads the file from the remote, but it does so at a very slow speed, even over a hardwired 1Gbps connection!

Increasing the buffer size for downloading these helps quite a bit.

Test setup:

  bash
  $ cat gen.py
  print('const char* hugeglobal = ')
  
  for _ in range(1000*500):
      print('  "' + '1234'*50 + '"')
  
  print(';')
  print('const char* mystring() { return hugeglobal; }')
  $ gen.py > huge.c
  $ mkdir libdir
  $ gcc -fPIC huge.c -Wl,-soname,libhuge.so -o libdir/libhuge.so -shared
  $ gcc test.c -L libdir -l huge -Wl,-rpath='$$ORIGIN' -o test
  $ rsync libdir remote:~/
  $ ssh remote bash -c "cd ~/libdir /llvm/buildr/bin/lldb-server platform --server --listen '*:1234'"

in another terminal

  bash
  $ rm -rf ~/.lldb # clear cache
  $ cat connect.lldb
  platform select remote-linux
  platform connect connect://10.0.0.14:1234
  file test
  b main
  r
  image list
  c
  q
  $ time /llvm/buildr/bin/lldb --source connect.lldb

Times with various buffer sizes:

1kiB (current): ~22s
8kiB: ~8s
16kiB: ~4s
32kiB: ~3.5s
64kiB: ~2.8s
128kiB: ~2.6s
256kiB: ~2.1s
512kiB: ~2.1s
1MiB: ~2.1s
2MiB: ~2.1s

I choose 512kiB from this list as it seems to be the place where the returns start diminishing and still isn't that much memory


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D153060

Files:
  lldb/source/Target/Platform.cpp


Index: lldb/source/Target/Platform.cpp
===================================================================
--- lldb/source/Target/Platform.cpp
+++ lldb/source/Target/Platform.cpp
@@ -1630,7 +1630,7 @@
     return error;
   }
 
-  std::vector<char> buffer(1024);
+  std::vector<char> buffer(512 * 1024);
   auto offset = src_offset;
   uint64_t total_bytes_read = 0;
   while (total_bytes_read < src_size) {


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D153060.531853.patch
Type: text/x-patch
Size: 407 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20230615/673f50b7/attachment.bin>