[Lldb-commits] [PATCH] D156375: Fix lldb-vscode frame id integer overflow

[jeffrey tan via Phabricator via lldb-commits]

yinghuitan created this revision.
yinghuitan added reviewers: clayborg, labath, jingham, jdoerfert, JDevlieghere, kusmour, GeorgeHuyubo.
Herald added a project: All.
yinghuitan requested review of this revision.
Herald added a project: LLDB.
Herald added a subscriber: lldb-commits.

This patch fixes a 32bit integer overflow in lldb-vscode.
The current implementation of frame_id does `(thread_index << 19 | frame_index)`. Since thread_index is a 32 bit integer this leaves only 32 - 19 == 13 bits available for the thread_index. As a result, lldb-vscode can only handle 2^13 == 8192 threads. Normally, this would be sufficient, but we have seen crazy process having +12000 threads, causing the frame_id algorithm above to integer overflow during casting.

The patch fixes the overflow by up casting to 64 bit integer first before bit shifiting.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D156375

Files:
  lldb/tools/lldb-vscode/LLDBUtils.cpp


Index: lldb/tools/lldb-vscode/LLDBUtils.cpp
===================================================================
--- lldb/tools/lldb-vscode/LLDBUtils.cpp
+++ lldb/tools/lldb-vscode/LLDBUtils.cpp
@@ -79,8 +79,8 @@
 }
 
 int64_t MakeVSCodeFrameID(lldb::SBFrame &frame) {
-  return (int64_t)(frame.GetThread().GetIndexID() << THREAD_INDEX_SHIFT |
-                   frame.GetFrameID());
+  return ((int64_t)frame.GetThread().GetIndexID() << THREAD_INDEX_SHIFT) |
+         frame.GetFrameID();
 }
 
 } // namespace lldb_vscode


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D156375.544529.patch
Type: text/x-patch
Size: 523 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20230726/cf241f02/attachment.bin>