<div dir="ltr">If you want to go with the path to implement it outside LLDB then I would suggest to implement it with an out of tree plugin written in C++. You can use the SB API the same way as you can from python and additionally it have a few advantages:<div>* You have a C/C++ API what makes it easy to integrate the functionality into an IDE (they just have to link to your shared library)</div><div>* You can generate a Python API if you need one with SWIG the same way we do it for the SB API</div><div>* You don't have to worry about making the code both Python 2.7 and Python 3.5 compatible</div><div><br></div><div>You can see a very simple example for implementing an out of tree C++ plugin in <lldb>/examples/plugins/commands</div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Feb 1, 2016 at 10:53 AM Pavel Labath via lldb-dev <<a href="mailto:lldb-dev@lists.llvm.org">lldb-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Speaking for Android Studio, I think that we *could* use a<br>
python-based implementation (hard to say exactly without knowing the<br>
details of the implementation), but I believe a different<br>
implementation could be *easier* to integrate. Plus, if the solution<br>
integrates more closely with lldb, we could surface some of the data<br>
in the command-line client as well.<br>
<br>
pl<br>
<br>
On 1 February 2016 at 10:30, Ravitheja Addepally<br>
<<a href="mailto:ravithejawork@gmail.com" target="_blank">ravithejawork@gmail.com</a>> wrote:<br>
> And what about the ease of integration into a an IDE, I don't really know if<br>
> the python based approach would be usable or not in this context ?<br>
><br>
> On Mon, Feb 1, 2016 at 11:17 AM, Pavel Labath <<a href="mailto:labath@google.com" target="_blank">labath@google.com</a>> wrote:<br>
>><br>
>> It feels to me that the python based approach could run into a dead<br>
>> end fairly quickly: a) you can only access the data when the target is<br>
>> stopped; b) the self-tracing means that the evaluation of these<br>
>> expressions would introduce noise in the data; c) overhead of all the<br>
>> extra packets(?).<br>
>><br>
>> So, I would be in favor of a lldb-server based approach. I'm not<br>
>> telling you that you shouldn't do that, but I don't think that's an<br>
>> approach I would take...<br>
>><br>
>> pl<br>
>><br>
>><br>
>> On 1 February 2016 at 08:58, Ravitheja Addepally<br>
>> <<a href="mailto:ravithejawork@gmail.com" target="_blank">ravithejawork@gmail.com</a>> wrote:<br>
>> > Ok, that is one option, but one of the aim for this activity is to make<br>
>> > the<br>
>> > data available for use by the IDE's like Android Studio or XCode or any<br>
>> > other that may want to display this information in its environment so<br>
>> > keeping that in consideration would the complete python based approach<br>
>> > be<br>
>> > useful ? or would providing LLDB api's to extract raw perf data from the<br>
>> > target be useful ?<br>
>> ><br>
>> > On Thu, Jan 21, 2016 at 10:00 PM, Greg Clayton <<a href="mailto:gclayton@apple.com" target="_blank">gclayton@apple.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> One thing to think about is you can actually just run an expression in<br>
>> >> the<br>
>> >> program that is being debugged without needing to change anything in<br>
>> >> the GDB<br>
>> >> remote server. So this can all be done via python commands and would<br>
>> >> require<br>
>> >> no changes to anything. So you can run an expression to enable the<br>
>> >> buffer.<br>
>> >> Since LLDB supports multiple line expression that can define their own<br>
>> >> local<br>
>> >> variables and local types. So the expression could be something like:<br>
>> >><br>
>> >> int perf_fd = (int)perf_event_open(...);<br>
>> >> struct PerfData<br>
>> >> {<br>
>> >> void *data;<br>
>> >> size_t size;<br>
>> >> };<br>
>> >> PerfData result = read_perf_data(perf_fd);<br>
>> >> result<br>
>> >><br>
>> >><br>
>> >> The result is then a structure that you can access from your python<br>
>> >> command (it will be a SBValue) and then you can read memory in order to<br>
>> >> get<br>
>> >> the perf data.<br>
>> >><br>
>> >> You can also split things up into multiple calls where you can run<br>
>> >> perf_event_open() on its own and return the file descriptor:<br>
>> >><br>
>> >> (int)perf_event_open(...)<br>
>> >><br>
>> >> This expression will return the file descriptor<br>
>> >><br>
>> >> Then you could allocate memory via the SBProcess:<br>
>> >><br>
>> >> (void *)malloc(1024);<br>
>> >><br>
>> >> The result of this expression will be the buffer that you use...<br>
>> >><br>
>> >> Then you can read 1024 bytes at a time into this newly created buffer.<br>
>> >><br>
>> >> So a solution that is completely done in python would be very<br>
>> >> attractive.<br>
>> >><br>
>> >> Greg<br>
>> >><br>
>> >><br>
>> >> > On Jan 21, 2016, at 7:04 AM, Ravitheja Addepally<br>
>> >> > <<a href="mailto:ravithejawork@gmail.com" target="_blank">ravithejawork@gmail.com</a>> wrote:<br>
>> >> ><br>
>> >> > Hello,<br>
>> >> > Regarding the questions in this thread please find the answers<br>
>> >> > -><br>
>> >> ><br>
>> >> > How are you going to present this information to the user? (I know<br>
>> >> > debugserver can report some performance data... Have you looked into<br>
>> >> > how that works? Do you plan to reuse some parts of that<br>
>> >> > infrastructure?) and How will you get the information from the server<br>
>> >> > to<br>
>> >> > the client?<br>
>> >> ><br>
>> >> > Currently I plan to show a list of instructions that have been<br>
>> >> > executed<br>
>> >> > so far, I saw the<br>
>> >> > implementation suggested by pavel, the already present infrastructure<br>
>> >> > is<br>
>> >> > a little bit lacking in terms of the needs of the<br>
>> >> > project, but I plan to follow a similar approach, i.e to extract the<br>
>> >> > raw<br>
>> >> > trace data by querying the server (which can use the<br>
>> >> > perf_event_open to get the raw trace data from the kernel) and<br>
>> >> > transport<br>
>> >> > it through gdb packets ( qXfer packets<br>
>> >> ><br>
>> >> ><br>
>> >> > <a href="https://sourceware.org/gdb/onlinedocs/gdb/Branch-Trace-Format.html#Branch-Trace-Format" rel="noreferrer" target="_blank">https://sourceware.org/gdb/onlinedocs/gdb/Branch-Trace-Format.html#Branch-Trace-Format</a>).<br>
>> >> > At the client side the raw trace data<br>
>> >> > could be passed on to python based command that could decode the<br>
>> >> > data.<br>
>> >> > This also eliminates the dependency of libipt since LLDB<br>
>> >> > would not decode the data itself.<br>
>> >> ><br>
>> >> > There is also the question of this third party library. Do we take a<br>
>> >> > hard dependency on libipt (probably a non-starter), or only use it if<br>
>> >> > it's<br>
>> >> > available (much better)?<br>
>> >> ><br>
>> >> > With the above mentioned way LLDB would not need the library, who<br>
>> >> > ever<br>
>> >> > wants to use the python command would have to install it separately<br>
>> >> > but LLDB<br>
>> >> > wont need it<br>
>> >> ><br>
>> >> > With the performance counters, the interface would still be<br>
>> >> > perf_event_open, so if there was a perf_wrapper in LLDB server then<br>
>> >> > it could<br>
>> >> > be reused to configure and use the<br>
>> >> > software performance counters as well, you would just need to pass<br>
>> >> > different attributes in the perf_event_open system call, plus I think<br>
>> >> > the<br>
>> >> > perf_wrapper could be reused to<br>
>> >> > get CoreSight information as well (see<br>
>> >> > <a href="https://lwn.net/Articles/664236/" rel="noreferrer" target="_blank">https://lwn.net/Articles/664236/</a><br>
>> >> > )<br>
>> >> ><br>
>> >> ><br>
>> >> > On Wed, Oct 21, 2015 at 8:57 PM, Greg Clayton <<a href="mailto:gclayton@apple.com" target="_blank">gclayton@apple.com</a>><br>
>> >> > wrote:<br>
>> >> > one main benefit to doing this externally is allow this to be done<br>
>> >> > remotely over any debugger connection. If you can run expressions to<br>
>> >> > enable/disable/setup the memory buffer/access the buffer contents,<br>
>> >> > then you<br>
>> >> > don't need to add code into the debugger to actually do this.<br>
>> >> ><br>
>> >> > Greg<br>
>> >> ><br>
>> >> > > On Oct 21, 2015, at 11:54 AM, Greg Clayton <<a href="mailto:gclayton@apple.com" target="_blank">gclayton@apple.com</a>><br>
>> >> > > wrote:<br>
>> >> > ><br>
>> >> > > IMHO the best way to provide this information is to implement<br>
>> >> > > reverse<br>
>> >> > > debugging packets in a GDB server (lldb-server). If you enable this<br>
>> >> > > feature<br>
>> >> > > via some packet to lldb-server, and that enables the gathering of<br>
>> >> > > data that<br>
>> >> > > keeps the last N instructions run by all threads in some buffer<br>
>> >> > > that gets<br>
>> >> > > overwritten. The lldb-server enables it and gives a buffer to the<br>
>> >> > > perf_event_interface(). Then clients can ask the lldb-server to<br>
>> >> > > step back in<br>
>> >> > > any thread. Only when the data is requested do we actually use the<br>
>> >> > > data to<br>
>> >> > > implement the reverse stepping.<br>
>> >> > ><br>
>> >> > > Another way to do this would be to use a python based command that<br>
>> >> > > can<br>
>> >> > > be added to any target that supports this. The plug-in could<br>
>> >> > > install a set<br>
>> >> > > of LLDB commands. To see how to create new lldb command line<br>
>> >> > > commands in<br>
>> >> > > python, see the section named "CREATE A NEW LLDB COMMAND USING A<br>
>> >> > > PYTHON<br>
>> >> > > FUNCTION" on the <a href="http://lldb.llvm.org/python-reference.html" rel="noreferrer" target="_blank">http://lldb.llvm.org/python-reference.html</a> web<br>
>> >> > > page.<br>
>> >> > ><br>
>> >> > > Then you can have some commands like:<br>
>> >> > ><br>
>> >> > > intel-pt-start<br>
>> >> > > intel-pt-dump<br>
>> >> > > intel-pt-stop<br>
>> >> > ><br>
>> >> > > Each command could have options and arguments as desired. The<br>
>> >> > > "intel-pt-start" command could make an expression call to enable<br>
>> >> > > the feature<br>
>> >> > > in the target by running and expression that runs the some<br>
>> >> > > perf_event_interface calls that would allocate some memory and hand<br>
>> >> > > it to<br>
>> >> > > the Intel PT stuff. The "intel-pt-dump" could just give a raw dump<br>
>> >> > > all of<br>
>> >> > > history for one or more threads (again, add options and arguments<br>
>> >> > > as needed<br>
>> >> > > to this command). The python code could bridge to C and use the<br>
>> >> > > intel<br>
>> >> > > libraries that know how to process the data.<br>
>> >> > ><br>
>> >> > > If this all goes well we can think about building it into LLDB as a<br>
>> >> > > built in command.<br>
>> >> > ><br>
>> >> > ><br>
>> >> > >> On Oct 21, 2015, at 9:50 AM, Zachary Turner via lldb-dev<br>
>> >> > >> <<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>> wrote:<br>
>> >> > >><br>
>> >> > >> There are two different kinds of performance counters: OS<br>
>> >> > >> performance<br>
>> >> > >> counters and CPU performance counters. It sounds like you're<br>
>> >> > >> talking about<br>
>> >> > >> the latter, but it's worth considering whether this could be<br>
>> >> > >> designed in a<br>
>> >> > >> way to support both (i.e. even if you don't do both yourself, at<br>
>> >> > >> least make<br>
>> >> > >> the machinery reusable and apply to both for when someone else<br>
>> >> > >> wanted to<br>
>> >> > >> come through and add OS perf counters).<br>
>> >> > >><br>
>> >> > >> There is also the question of this third party library. Do we<br>
>> >> > >> take a<br>
>> >> > >> hard dependency on libipt (probably a non-starter), or only use it<br>
>> >> > >> if it's<br>
>> >> > >> available (much better)?<br>
>> >> > >><br>
>> >> > >> As Pavel said, how are you planning to present the information to<br>
>> >> > >> the<br>
>> >> > >> user? Through some sort of top level command like "perfcount<br>
>> >> > >> instructions_retired"?<br>
>> >> > >><br>
>> >> > >> On Wed, Oct 21, 2015 at 8:16 AM Pavel Labath via lldb-dev<br>
>> >> > >> <<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>> wrote:<br>
>> >> > >> [ Moving this discussion back to the list. I pressed the wrong<br>
>> >> > >> button<br>
>> >> > >> when replying.]<br>
>> >> > >><br>
>> >> > >> Thanks for the explanation Ravi. It sounds like a very useful<br>
>> >> > >> feature<br>
>> >> > >> indeed. I've found a reference to the debugserver profile data in<br>
>> >> > >> GDBRemoteCommunicationClient.cpp:1276, so maybe that will help<br>
>> >> > >> with<br>
>> >> > >> your investigation. Maybe also someone more knowledgeable can<br>
>> >> > >> explain<br>
>> >> > >> what those A packets are used for (?).<br>
>> >> > >><br>
>> >> > >><br>
>> >> > >> On 21 October 2015 at 15:48, Ravitheja Addepally<br>
>> >> > >> <<a href="mailto:ravithejawork@gmail.com" target="_blank">ravithejawork@gmail.com</a>> wrote:<br>
>> >> > >>> Hi,<br>
>> >> > >>> Thanx for your reply, some of the future processors to be<br>
>> >> > >>> released<br>
>> >> > >>> by<br>
>> >> > >>> Intel have this hardware support for recording the instructions<br>
>> >> > >>> that<br>
>> >> > >>> were<br>
>> >> > >>> executed by the processor and this recording process is also<br>
>> >> > >>> quite<br>
>> >> > >>> fast and<br>
>> >> > >>> does not add too much computational load. Now this hardware is<br>
>> >> > >>> made<br>
>> >> > >>> accessible via the perf_event_interface where one could map a<br>
>> >> > >>> region<br>
>> >> > >>> of<br>
>> >> > >>> memory for this purpose by passing it as an argument to this<br>
>> >> > >>> perf_event_interface. The recorded instructions are then written<br>
>> >> > >>> to<br>
>> >> > >>> the<br>
>> >> > >>> memory region assigned. Now this is basically the raw<br>
>> >> > >>> information,<br>
>> >> > >>> which can<br>
>> >> > >>> be obtained from the hardware. It can be interpreted and<br>
>> >> > >>> presented<br>
>> >> > >>> to the<br>
>> >> > >>> user in the following ways -><br>
>> >> > >>><br>
>> >> > >>> 1) Instruction history - where the user gets basically a list of<br>
>> >> > >>> all<br>
>> >> > >>> instructions that were executed<br>
>> >> > >>> 2) Function Call History - It is also possible to get a list of<br>
>> >> > >>> all<br>
>> >> > >>> the<br>
>> >> > >>> functions called in the inferior<br>
>> >> > >>> 3) Reverse Debugging with limited information - In GDB this is<br>
>> >> > >>> only<br>
>> >> > >>> the<br>
>> >> > >>> functions executed.<br>
>> >> > >>><br>
>> >> > >>> This raw information also needs to decoded (even before you can<br>
>> >> > >>> disassemble<br>
>> >> > >>> it ), there is already a library released by Intel called libipt<br>
>> >> > >>> which can<br>
>> >> > >>> do that. At the moment we plan to work with Instruction History.<br>
>> >> > >>> I will look into the debugserver infrastructure and get back to<br>
>> >> > >>> you.<br>
>> >> > >>> I guess<br>
>> >> > >>> for the server client communication we would rely on packets<br>
>> >> > >>> only.<br>
>> >> > >>> In case<br>
>> >> > >>> of concerns about too much data being transferred, we can limit<br>
>> >> > >>> the<br>
>> >> > >>> number<br>
>> >> > >>> of entries we report because anyway the amount of data recorded<br>
>> >> > >>> is<br>
>> >> > >>> too big<br>
>> >> > >>> to present all at once so we would have to resort to something<br>
>> >> > >>> like<br>
>> >> > >>> a<br>
>> >> > >>> viewport.<br>
>> >> > >>><br>
>> >> > >>> Since a lot of instructions can be recorded this way, the<br>
>> >> > >>> function<br>
>> >> > >>> call<br>
>> >> > >>> history can be quite useful for debugging and especially since it<br>
>> >> > >>> is<br>
>> >> > >>> a lot<br>
>> >> > >>> faster to collect function traces this way.<br>
>> >> > >>><br>
>> >> > >>> -ravi<br>
>> >> > >>><br>
>> >> > >>> On Wed, Oct 21, 2015 at 3:14 PM, Pavel Labath <<a href="mailto:labath@google.com" target="_blank">labath@google.com</a>><br>
>> >> > >>> wrote:<br>
>> >> > >>>><br>
>> >> > >>>> Hi,<br>
>> >> > >>>><br>
>> >> > >>>> I am not really familiar with the perf_event interface (and I<br>
>> >> > >>>> suspect<br>
>> >> > >>>> others aren't also), so it might help if you explain what kind<br>
>> >> > >>>> of<br>
>> >> > >>>> information do you plan to collect from there.<br>
>> >> > >>>><br>
>> >> > >>>> As for the PtraceWrapper question, I think that really depends<br>
>> >> > >>>> on<br>
>> >> > >>>> bigger design decisions. My two main questions for a feature<br>
>> >> > >>>> like<br>
>> >> > >>>> this<br>
>> >> > >>>> would be:<br>
>> >> > >>>> - How are you going to present this information to the user? (I<br>
>> >> > >>>> know<br>
>> >> > >>>> debugserver can report some performance data... Have you looked<br>
>> >> > >>>> into<br>
>> >> > >>>> how that works? Do you plan to reuse some parts of that<br>
>> >> > >>>> infrastructure?)<br>
>> >> > >>>> - How will you get the information from the server to the<br>
>> >> > >>>> client?<br>
>> >> > >>>><br>
>> >> > >>>> pl<br>
>> >> > >>>><br>
>> >> > >>>><br>
>> >> > >>>> On 21 October 2015 at 13:41, Ravitheja Addepally via lldb-dev<br>
>> >> > >>>> <<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a>> wrote:<br>
>> >> > >>>>> Hello,<br>
>> >> > >>>>> I want to implement support for reading Performance<br>
>> >> > >>>>> measurement<br>
>> >> > >>>>> information using the perf_event_open system calls. The motive<br>
>> >> > >>>>> is<br>
>> >> > >>>>> to add<br>
>> >> > >>>>> support for Intel PT hardware feature, which is available<br>
>> >> > >>>>> through<br>
>> >> > >>>>> the<br>
>> >> > >>>>> perf_event interface. I was thinking of implementing a new<br>
>> >> > >>>>> Wrapper<br>
>> >> > >>>>> like<br>
>> >> > >>>>> PtraceWrapper in NativeProcessLinux files. My query is that, is<br>
>> >> > >>>>> this a<br>
>> >> > >>>>> correct place to start or not ? in case not, could someone<br>
>> >> > >>>>> suggest<br>
>> >> > >>>>> me<br>
>> >> > >>>>> another place to begin with ?<br>
>> >> > >>>>><br>
>> >> > >>>>> BR,<br>
>> >> > >>>>> A Ravi Theja<br>
>> >> > >>>>><br>
>> >> > >>>>><br>
>> >> > >>>>> _______________________________________________<br>
>> >> > >>>>> lldb-dev mailing list<br>
>> >> > >>>>> <a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a><br>
>> >> > >>>>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
>> >> > >>>>><br>
>> >> > >>><br>
>> >> > >>><br>
>> >> > >> _______________________________________________<br>
>> >> > >> lldb-dev mailing list<br>
>> >> > >> <a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a><br>
>> >> > >> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
>> >> > >> _______________________________________________<br>
>> >> > >> lldb-dev mailing list<br>
>> >> > >> <a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a><br>
>> >> > >> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
>> >> > ><br>
>> >> ><br>
>> >> ><br>
>> >><br>
>> ><br>
><br>
><br>
_______________________________________________<br>
lldb-dev mailing list<br>
<a href="mailto:lldb-dev@lists.llvm.org" target="_blank">lldb-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br>
</blockquote></div>