[llvm-dev] Rewriting opt-viewer in C++

[Bob Haarman via llvm-dev]

Thanks for your comments, everyone! I'll try to answer the questions people
have asked. First, let me say that I like Python, so I would be happy to
keep the tool in Python if people feel that is a better way to go and we
can still get it to go fast. As for precedent, we have several Python
scripts already shipping with LLVM, including lit, and Python is listed in
the requirements, so I imagine it's fine from that perspective.

Having said that, the C++ version is quite a lot faster. I'm seeing about a
60x difference in how long it takes to generate HTML. This is with the C++
version doing a bit more work - specifically, it correctly encodes
characters that have special meaning to HTML. I've been testing this on a
build of the Chromium web browser with reports for the inline pass. On my
machine, processing only the storage directory takes about 7 seconds with
the C++ version and about 6 minutes and 20 seconds with the Python version.
The v8 directory takes about 35 seconds with the C++ version and 44 minutes
with the Python version. Processing the whole build takes about 13 minutes
with the C++ version; when I tried it with the Python version over night,
it was still going 14 hours later.

The way I would like to use the tool is to perform a compilation and then
look at the optimization report to look at which optimizations were missed
and why, then change the code and/or the compiler to see if we can get
faster code to be emitted. Ideally, I would like to have the optimization
report ready for viewing soon after the compilation. Being able to generate
the report in seconds, or less than a second, is definitely more useful to
my use case than taking minutes. I figured we might get a good speedup from
rewriting the tool in C++, and if we ever were going to do that, it would
be easier to do it when the tool was relatively simple, as it is now. I
believe my rewrite is complete now, and I think the increase in speed
actually makes the tool more useful in that it can now feasibly be used on
larger projects. I'll clean up the code a bit (it's very much proof of
concept now) and put it up for review once that's done.

Bob



On Mon, Nov 14, 2016 at 9:12 AM, Adam Nemet <anemet at apple.com> wrote:

> Again I am still undecided which way this should go but I was also
> wondering about the speed difference if we used the C-based parser in
> PyYAML (http://pyyaml.org/wiki/LibYAML).
>
> On Nov 13, 2016, at 12:19 AM, Adam Nemet <anemet at apple.com> wrote:
>
> Hi Bob,
>
> I am glad you’re finding opt-viewer useful.  I am generally fine this
> being rewritten in C++.  The idea was that once the prototype proves itself
> to be useful this would be considered.
>
> Do you have ideas how to get syntax highlighting implemented?  In Python,
> Pygments makes this easy.
>
> Another usage model I was considering is for people who ssh to remote
> servers for their builds.  In this case it should be easy in Python to have
>  a simple HTTP server started in opt-viewer.
>
> On the other hand, I am not sure if we have precedence to ship tools
> written in Python in LLVM so we may *have to* move.
>
> Adam
>
> On Nov 11, 2016, at 11:50 PM, Bob Haarman <llvm at inglorion.net> wrote:
>
> Hi Adam,
>
> Thank you for writing opt-viewer! I've been playing around with it and
> found it helps me understand what optimizations were performed, which were
> missed, and why, as well as easily navigate to the relevant code. Wanting
> to use it on larger projects, I found that it can take a while to analyze
> the optimization reports. Since there doesn't seem to be too much code in
> it yet, I figured it wouldn't be too much work to implement the same
> functionality in C++, which should give us a nice speedup compared to the
> current Python implementation. I have something that implements most of the
> functionality: the index page is generated and links to lines on source
> pages, which are also generated, but I haven't gotten to annotations on
> source pages yet. I expect to put up some code for review next week, but
> wanted to post here in case others are thinking along similar lines so we
> can hopefully avoid duplicating effort.
>
> Cheers,
>
> Bob
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20161115/08840d24/attachment.html>