[all-commits] [llvm/llvm-project] 1a3f99: [trace][intelpt] Support system-wide tracing [13] ...
walter erquinigo via All-commits
all-commits at lists.llvm.org
Thu Jun 16 11:23:21 PDT 2022
Branch: refs/heads/main
Home: https://github.com/llvm/llvm-project
Commit: 1a3f996972b175a6f7e4db1eabb8310e2e93f634
https://github.com/llvm/llvm-project/commit/1a3f996972b175a6f7e4db1eabb8310e2e93f634
Author: Walter Erquinigo <wallace at fb.com>
Date: 2022-06-16 (Thu, 16 Jun 2022)
Changed paths:
M lldb/include/lldb/Target/Trace.h
M lldb/include/lldb/Utility/TraceGDBRemotePackets.h
M lldb/include/lldb/Utility/TraceIntelPTGDBRemotePackets.h
M lldb/source/Plugins/Process/Linux/Perf.cpp
M lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationClient.cpp
M lldb/source/Plugins/Trace/intel-pt/CMakeLists.txt
M lldb/source/Plugins/Trace/intel-pt/CommandObjectTraceStartIntelPT.h
M lldb/source/Plugins/Trace/intel-pt/TaskTimer.cpp
M lldb/source/Plugins/Trace/intel-pt/TaskTimer.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTJSONStructs.cpp
A lldb/source/Plugins/Trace/intel-pt/TraceIntelPTMultiCoreDecoder.cpp
A lldb/source/Plugins/Trace/intel-pt/TraceIntelPTMultiCoreDecoder.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionFileParser.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionSaver.cpp
M lldb/source/Target/Trace.cpp
M lldb/source/Utility/TraceGDBRemotePackets.cpp
M lldb/source/Utility/TraceIntelPTGDBRemotePackets.cpp
M lldb/test/API/commands/trace/TestTraceDumpInfo.py
M lldb/test/API/commands/trace/TestTraceLoad.py
M lldb/test/API/commands/trace/multiple-threads/TestTraceStartStopMultipleThreads.py
Log Message:
-----------
[trace][intelpt] Support system-wide tracing [13] - Add context switch decoding
- Add the logic that parses all cpu context switch traces and produces blocks of continuous executions, which will be later used to assign intel pt subtraces to threads and to identify gaps. This logic can also identify if the context switch trace is malformed.
- The continuous executions blocks are able to indicate when there were some contention issues when producing the context switch trace. See the inline comments for more information.
- Update the 'dump info' command to show information and stats related to the multicore decoding flow, including timing about context switch decoding.
- Add the logic to conver nanoseconds to TSCs.
- Fix a bug when returning the context switches. Now they data returned makes sense and even empty traces can be returned from lldb-server.
- Finish the necessary bits for loading and saving a multi-core trace bundle from disk.
- Change some size_t to uint64_t for compatibility with 32 bit systems.
Tested by saving a trace session of a program that sleeps 100 times, it was able to produce the following 'dump info' text:
```
(lldb) trace load /tmp/trace3/trace.json (lldb) thread trace dump info Trace technology: intel-pt
thread #1: tid = 4192415
Total number of instructions: 1
Memory usage:
Total approximate memory usage (excluding raw trace): 2.51 KiB
Average memory usage per instruction (excluding raw trace): 2573.00 bytes
Timing for this thread:
Timing for global tasks:
Context switch trace decoding: 0.00s
Events:
Number of instructions with events: 0
Number of individual events: 0
Multi-core decoding:
Total number of continuous executions found: 2499
Number of continuous executions for this thread: 102
Errors:
Number of TSC decoding errors: 0
```
Differential Revision: https://reviews.llvm.org/D126267
Commit: a19fcc2bec81989f90700cfc63b89a0dfd330197
https://github.com/llvm/llvm-project/commit/a19fcc2bec81989f90700cfc63b89a0dfd330197
Author: Walter Erquinigo <wallace at fb.com>
Date: 2022-06-16 (Thu, 16 Jun 2022)
Changed paths:
M lldb/include/lldb/Target/Trace.h
M lldb/include/lldb/Utility/TraceIntelPTGDBRemotePackets.h
M lldb/source/Plugins/Trace/intel-pt/CMakeLists.txt
M lldb/source/Plugins/Trace/intel-pt/DecodedThread.h
M lldb/source/Plugins/Trace/intel-pt/LibiptDecoder.cpp
M lldb/source/Plugins/Trace/intel-pt/LibiptDecoder.h
A lldb/source/Plugins/Trace/intel-pt/PerfContextSwitchDecoder.cpp
A lldb/source/Plugins/Trace/intel-pt/PerfContextSwitchDecoder.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTJSONStructs.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTMultiCoreDecoder.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTMultiCoreDecoder.h
M lldb/source/Target/Trace.cpp
M lldb/source/Utility/TraceIntelPTGDBRemotePackets.cpp
M lldb/test/API/commands/trace/TestTraceLoad.py
A lldb/test/API/commands/trace/intelpt-multi-core-trace/cores/45.intelpt_trace
A lldb/test/API/commands/trace/intelpt-multi-core-trace/cores/45.perf_context_switch_trace
A lldb/test/API/commands/trace/intelpt-multi-core-trace/cores/51.intelpt_trace
A lldb/test/API/commands/trace/intelpt-multi-core-trace/cores/51.perf_context_switch_trace
A lldb/test/API/commands/trace/intelpt-multi-core-trace/modules/m.out
A lldb/test/API/commands/trace/intelpt-multi-core-trace/multi_thread.cpp
A lldb/test/API/commands/trace/intelpt-multi-core-trace/trace.json
M lldb/test/API/commands/trace/multiple-threads/TestTraceStartStopMultipleThreads.py
M lldb/unittests/Process/Linux/PerfTests.cpp
Log Message:
-----------
[trace][intelpt] Support system-wide tracing [14] - Decode per cpu
This is the final functional patch to support intel pt decoding per cpu.
It works by doing the following:
- First, all context switches are split by tid and sorted in order. This produces a list of continuous executes per thread per core.
- Then, all intel pt subtraces are split by PSB boundaries and assigned to individual thread continuous executions on the same core by doing simple TSC-based comparisons.
- With this, we have, per thread, a sorted list of continuous executions each one with a list of intel pt subtraces. Up to this point, this is really fast because no instructions were actually decoded.
- Then, each thread can be decoded by traversing their continuous executions and intel pt subtraces. An advantage of having these continuous executions is that we can identify if a continuous exexecution doesn't have intel pt data, and thus has a gap in it. We can later to more sofisticated comparisons to identify if within a continuous execution there are gaps.
I'm adding a test as well.
Differential Revision: https://reviews.llvm.org/D126394
Commit: ef9970759b5b63fed40061cd960e6d9e8edabb48
https://github.com/llvm/llvm-project/commit/ef9970759b5b63fed40061cd960e6d9e8edabb48
Author: Walter Erquinigo <wallace at fb.com>
Date: 2022-06-16 (Thu, 16 Jun 2022)
Changed paths:
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTJSONStructs.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionFileParser.cpp
M lldb/test/API/commands/trace/TestTraceLoad.py
M lldb/test/API/commands/trace/intelpt-multi-core-trace/trace.json
M lldb/test/API/commands/trace/intelpt-trace/trace_bad2.json
Log Message:
-----------
[trace][intelpt] Support system-wide tracing [15] - Make triple optional
The process triple should only be needed when LLDB can't identify the correct
triple on its own. Examples could be universal mach-o binaries. But in any case,
at least for most of ELF files, LLDB should be able to do the job without having
the user specify the triple manually.
Differential Revision: https://reviews.llvm.org/D126990
Commit: ff15efc1a701a8e76242ca2a105b5adf7d68e980
https://github.com/llvm/llvm-project/commit/ff15efc1a701a8e76242ca2a105b5adf7d68e980
Author: Walter Erquinigo <wallace at fb.com>
Date: 2022-06-16 (Thu, 16 Jun 2022)
Changed paths:
M lldb/include/lldb/Target/Trace.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionFileParser.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionFileParser.h
M lldb/source/Target/Trace.cpp
M lldb/test/API/commands/trace/TestTraceLoad.py
A lldb/test/API/commands/trace/intelpt-multi-core-trace/trace_missing_threads.json
Log Message:
-----------
[trace][intelpt] Support system-wide tracing [16] - Create threads automatically from context switch data in the post-mortem case
For some context, The context switch data contains information of which threads were
executed by each traced process, therefore it's not necessary to specify
them in the trace file.
So this diffs adds support for that automatic feature. Eventually we
could include it to live processes as well.
Differential Revision: https://reviews.llvm.org/D127001
Commit: 03cc58ff2a7a88b4eb8c1409169b662b46ec85d9
https://github.com/llvm/llvm-project/commit/03cc58ff2a7a88b4eb8c1409169b662b46ec85d9
Author: Walter Erquinigo <wallace at fb.com>
Date: 2022-06-16 (Thu, 16 Jun 2022)
Changed paths:
M lldb/include/lldb/Target/Trace.h
M lldb/source/Plugins/Process/Linux/IntelPTCollector.cpp
M lldb/source/Plugins/Process/Linux/IntelPTCollector.h
M lldb/source/Plugins/Process/Linux/IntelPTMultiCoreTrace.cpp
M lldb/source/Plugins/Process/Linux/IntelPTMultiCoreTrace.h
M lldb/source/Plugins/Process/Linux/IntelPTPerThreadProcessTrace.cpp
M lldb/source/Plugins/Process/Linux/IntelPTPerThreadProcessTrace.h
M lldb/source/Plugins/Process/Linux/IntelPTProcessTrace.h
M lldb/source/Plugins/Process/Linux/IntelPTSingleBufferTrace.h
M lldb/source/Plugins/Process/Linux/Perf.cpp
M lldb/source/Plugins/Process/Linux/Perf.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPT.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTJSONStructs.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTMultiCoreDecoder.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTMultiCoreDecoder.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionFileParser.cpp
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionFileParser.h
M lldb/source/Plugins/Trace/intel-pt/TraceIntelPTSessionSaver.cpp
M lldb/source/Target/Trace.cpp
M lldb/test/API/commands/trace/intelpt-multi-core-trace/trace.json
M lldb/test/API/commands/trace/intelpt-multi-core-trace/trace_missing_threads.json
Log Message:
-----------
[trace][intelpt] Support system-wide tracing [17] - Some improvements
This improves several things and addresses comments up to the diff [11] in this stack.
- Simplify many functions to receive less parameters that they can identify easily
- Create Storage classes for Trace and TraceIntelPT that can make it easier to reason about what can change with live process refreshes and what cannot.
- Don't cache the perf zero conversion numbers in lldb-server to make sure we get the most up-to-date numbers.
- Move the thread identifaction from context switches to the bundle parser, to leave TraceIntelPT simpler. This also makes sure that the constructor of TraceIntelPT is invoked when the entire data has been checked to be correct.
- Normalize all bundle paths before the Processes, Threads and Modules are created, so that they can assume that all paths are correct and absolute
- Fix some issues in the tests. Now they all pass.
- return the specific instance when constructing PerThread and MultiCore processor tracers.
- Properly implement IntelPTMultiCoreTrace::TraceStart.
- Improve some comments.
- Use the typedef ContextSwitchTrace more often for clarity.
- Move CreateContextSwitchTracePerfEvent to Perf.h as a utility function.
- Synchronize better the state of the context switch and the intel pt
perf events.
- Use a booblean instead of an enum for the PerfEvent state.
Differential Revision: https://reviews.llvm.org/D127456
Compare: https://github.com/llvm/llvm-project/compare/ff0122dccea8...03cc58ff2a7a
More information about the All-commits
mailing list