[llvm-dev] Rewriting opt-viewer in C++

[Bob Haarman via llvm-dev]

That's compared to the implementation with the Python parser. So if the
libYAML parser is 6x the speed of that, the C++ version would be about 10x
the speed of the implementation with libYAML, instead of 60x.

On Tue, Nov 15, 2016 at 10:50 AM, Adam Nemet <anemet at apple.com> wrote:

>
> On Nov 15, 2016, at 10:33 AM, Bob Haarman <inglorion at google.com> wrote:
>
> Thanks for your comments, everyone! I'll try to answer the questions
> people have asked. First, let me say that I like Python, so I would be
> happy to keep the tool in Python if people feel that is a better way to go
> and we can still get it to go fast. As for precedent, we have several
> Python scripts already shipping with LLVM, including lit, and Python is
> listed in the requirements, so I imagine it's fine from that perspective.
>
> Having said that, the C++ version is quite a lot faster. I'm seeing about
> a 60x difference in how long it takes to generate HTML. This is with the
> C++ version doing a bit more work - specifically, it correctly encodes
> characters that have special meaning to HTML. I've been testing this on a
> build of the Chromium web browser with reports for the inline pass. On my
> machine, processing only the storage directory takes about 7 seconds with
> the C++ version and about 6 minutes and 20 seconds with the Python version.
> The v8 directory takes about 35 seconds with the C++ version and 44 minutes
> with the Python version. Processing the whole build takes about 13 minutes
> with the C++ version; when I tried it with the Python version over night,
> it was still going 14 hours later.
>
>
> Is this compared against the libYAML parser or the original Python
> parser?  See my mail from last night.
>
> Adam
>
> The way I would like to use the tool is to perform a compilation and then
> look at the optimization report to look at which optimizations were missed
> and why, then change the code and/or the compiler to see if we can get
> faster code to be emitted. Ideally, I would like to have the optimization
> report ready for viewing soon after the compilation. Being able to generate
> the report in seconds, or less than a second, is definitely more useful to
> my use case than taking minutes. I figured we might get a good speedup from
> rewriting the tool in C++, and if we ever were going to do that, it would
> be easier to do it when the tool was relatively simple, as it is now. I
> believe my rewrite is complete now, and I think the increase in speed
> actually makes the tool more useful in that it can now feasibly be used on
> larger projects. I'll clean up the code a bit (it's very much proof of
> concept now) and put it up for review once that's done.
>
> Bob
>
>
>
> On Mon, Nov 14, 2016 at 9:12 AM, Adam Nemet <anemet at apple.com> wrote:
>
>> Again I am still undecided which way this should go but I was also
>> wondering about the speed difference if we used the C-based parser in
>> PyYAML (http://pyyaml.org/wiki/LibYAML).
>>
>> On Nov 13, 2016, at 12:19 AM, Adam Nemet <anemet at apple.com> wrote:
>>
>> Hi Bob,
>>
>> I am glad you’re finding opt-viewer useful.  I am generally fine this
>> being rewritten in C++.  The idea was that once the prototype proves itself
>> to be useful this would be considered.
>>
>> Do you have ideas how to get syntax highlighting implemented?  In Python,
>> Pygments makes this easy.
>>
>> Another usage model I was considering is for people who ssh to remote
>> servers for their builds.  In this case it should be easy in Python to have
>>  a simple HTTP server started in opt-viewer.
>>
>> On the other hand, I am not sure if we have precedence to ship tools
>> written in Python in LLVM so we may *have to* move.
>>
>> Adam
>>
>> On Nov 11, 2016, at 11:50 PM, Bob Haarman <llvm at inglorion.net> wrote:
>>
>> Hi Adam,
>>
>> Thank you for writing opt-viewer! I've been playing around with it and
>> found it helps me understand what optimizations were performed, which were
>> missed, and why, as well as easily navigate to the relevant code. Wanting
>> to use it on larger projects, I found that it can take a while to analyze
>> the optimization reports. Since there doesn't seem to be too much code in
>> it yet, I figured it wouldn't be too much work to implement the same
>> functionality in C++, which should give us a nice speedup compared to the
>> current Python implementation. I have something that implements most of the
>> functionality: the index page is generated and links to lines on source
>> pages, which are also generated, but I haven't gotten to annotations on
>> source pages yet. I expect to put up some code for review next week, but
>> wanted to post here in case others are thinking along similar lines so we
>> can hopefully avoid duplicating effort.
>>
>> Cheers,
>>
>> Bob
>>
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20161115/ea2ef445/attachment-0001.html>