[llvm-dev] Rewriting opt-viewer in C++

[Bob Haarman via llvm-dev]

I've put up the code for my implementation at
https://reviews.llvm.org/D26723 so people can review it and try it for
themselves.

On Tue, Nov 15, 2016 at 5:55 PM, Bob Haarman <inglorion at google.com> wrote:

> That's compared to the implementation with the Python parser. So if the
> libYAML parser is 6x the speed of that, the C++ version would be about 10x
> the speed of the implementation with libYAML, instead of 60x.
>
> On Tue, Nov 15, 2016 at 10:50 AM, Adam Nemet <anemet at apple.com> wrote:
>
>>
>> On Nov 15, 2016, at 10:33 AM, Bob Haarman <inglorion at google.com> wrote:
>>
>> Thanks for your comments, everyone! I'll try to answer the questions
>> people have asked. First, let me say that I like Python, so I would be
>> happy to keep the tool in Python if people feel that is a better way to go
>> and we can still get it to go fast. As for precedent, we have several
>> Python scripts already shipping with LLVM, including lit, and Python is
>> listed in the requirements, so I imagine it's fine from that perspective.
>>
>> Having said that, the C++ version is quite a lot faster. I'm seeing about
>> a 60x difference in how long it takes to generate HTML. This is with the
>> C++ version doing a bit more work - specifically, it correctly encodes
>> characters that have special meaning to HTML. I've been testing this on a
>> build of the Chromium web browser with reports for the inline pass. On my
>> machine, processing only the storage directory takes about 7 seconds with
>> the C++ version and about 6 minutes and 20 seconds with the Python version.
>> The v8 directory takes about 35 seconds with the C++ version and 44 minutes
>> with the Python version. Processing the whole build takes about 13 minutes
>> with the C++ version; when I tried it with the Python version over night,
>> it was still going 14 hours later.
>>
>>
>> Is this compared against the libYAML parser or the original Python
>> parser?  See my mail from last night.
>>
>> Adam
>>
>> The way I would like to use the tool is to perform a compilation and then
>> look at the optimization report to look at which optimizations were missed
>> and why, then change the code and/or the compiler to see if we can get
>> faster code to be emitted. Ideally, I would like to have the optimization
>> report ready for viewing soon after the compilation. Being able to generate
>> the report in seconds, or less than a second, is definitely more useful to
>> my use case than taking minutes. I figured we might get a good speedup from
>> rewriting the tool in C++, and if we ever were going to do that, it would
>> be easier to do it when the tool was relatively simple, as it is now. I
>> believe my rewrite is complete now, and I think the increase in speed
>> actually makes the tool more useful in that it can now feasibly be used on
>> larger projects. I'll clean up the code a bit (it's very much proof of
>> concept now) and put it up for review once that's done.
>>
>> Bob
>>
>>
>>
>> On Mon, Nov 14, 2016 at 9:12 AM, Adam Nemet <anemet at apple.com> wrote:
>>
>>> Again I am still undecided which way this should go but I was also
>>> wondering about the speed difference if we used the C-based parser in
>>> PyYAML (http://pyyaml.org/wiki/LibYAML).
>>>
>>> On Nov 13, 2016, at 12:19 AM, Adam Nemet <anemet at apple.com> wrote:
>>>
>>> Hi Bob,
>>>
>>> I am glad you’re finding opt-viewer useful.  I am generally fine this
>>> being rewritten in C++.  The idea was that once the prototype proves itself
>>> to be useful this would be considered.
>>>
>>> Do you have ideas how to get syntax highlighting implemented?  In
>>> Python, Pygments makes this easy.
>>>
>>> Another usage model I was considering is for people who ssh to remote
>>> servers for their builds.  In this case it should be easy in Python to have
>>>  a simple HTTP server started in opt-viewer.
>>>
>>> On the other hand, I am not sure if we have precedence to ship tools
>>> written in Python in LLVM so we may *have to* move.
>>>
>>> Adam
>>>
>>> On Nov 11, 2016, at 11:50 PM, Bob Haarman <llvm at inglorion.net> wrote:
>>>
>>> Hi Adam,
>>>
>>> Thank you for writing opt-viewer! I've been playing around with it and
>>> found it helps me understand what optimizations were performed, which were
>>> missed, and why, as well as easily navigate to the relevant code. Wanting
>>> to use it on larger projects, I found that it can take a while to analyze
>>> the optimization reports. Since there doesn't seem to be too much code in
>>> it yet, I figured it wouldn't be too much work to implement the same
>>> functionality in C++, which should give us a nice speedup compared to the
>>> current Python implementation. I have something that implements most of the
>>> functionality: the index page is generated and links to lines on source
>>> pages, which are also generated, but I haven't gotten to annotations on
>>> source pages yet. I expect to put up some code for review next week, but
>>> wanted to post here in case others are thinking along similar lines so we
>>> can hopefully avoid duplicating effort.
>>>
>>> Cheers,
>>>
>>> Bob
>>>
>>>
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20161115/f5b9f691/attachment-0001.html>