[llvm-dev] RFC: Extending optimization reporting

via llvm-dev llvm-dev at lists.llvm.org
Thu May 23 06:27:34 PDT 2019

Hi Andy,

"[Robert Cox] was a little nervous about depending on debug information because he’s seen some things in the past where that changed how things were optimized."
Any such case should be considered a bug, and reported. There used to be some really egregious cases, lately we've found more isolated instances. There's only one situation I'm aware of that we let ride: Generating call-frame information introduces scheduling barriers.  Note that the source-location information has never (AFAIK) affected optimization, it's the more invasive value-tracking stuff that tends to be the problem.

"I guess with the way debug info is implemented in LLVM there’s a more or less continuous spectrum from just keep a couple of pieces of metadata to track source location through to full DWARF support,"
There is a source-location-only mode intended to support optimization remarks, i.e. be able to associate a source location with a remark that you want to report.  Offhand I don't recall whether that mode supports tracking inlined scopes, but it probably could if it doesn't now.  One of my team recently fixed a bug with updating the source location on loops in a function being inlined, so we should have that distinction available now.  I don't think there's anything in place that distinguishes copies of an unrolled loop though.

"Speaking for myself I’m also a little concerned about the tendency of debug info to get dropped during optimization, but I guess that doesn’t happen as regularly as it used to?"
Dropping location info is frequently a bug, although it's not as hard-and-fast a rule as the one about not affecting optimization (sometimes there just isn't a reasonable location to use; DWARF does not support associating two source locations with the same instruction).  I know my team tries to fix these whenever we find them, because we are *very* interested in improving the quality of the debugging experience on optimized code.  If you run into cases with losing/poor-quality optimization remarks, likely there is a poor debugging experience to match, so a bug report (if not a patch) would benefit everyone.


From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Kaylor, Andrew via llvm-dev
Sent: Wednesday, May 22, 2019 5:53 PM
To: anemet at apple.com
Cc: llvm-dev; Francis Visoiu Mistrih
Subject: Re: [llvm-dev] RFC: Extending optimization reporting

Hi Adam,

I don’t have much to report here, but I wanted to let you know that I haven’t forgotten about this completely.

I talked to Robert Cox about the inlining part of things, and he agreed that at a high level the problem we want to solve is essentially like the loop versioning problem in that we mostly need some kind of ID and a place to hang it. He was a little nervous about depending on debug information because he’s seen some things in the past where that changed how things were optimized. I guess with the way debug info is implemented in LLVM there’s a more or less continuous spectrum from just keep a couple of pieces of metadata to track source location through to full DWARF support, so any attempt to solve this with metadata that isn’t really debug info would be pointless and redundant. Speaking for myself I’m also a little concerned about the tendency of debug info to get dropped during optimization, but I guess that doesn’t happen as regularly as it used to?

I hope to be able to do more with this soon.


From: anemet at apple.com <anemet at apple.com>
Sent: Tuesday, May 14, 2019 9:49 AM
To: Kaylor, Andrew <andrew.kaylor at intel.com>
Cc: llvm-dev <llvm-dev at lists.llvm.org>; Francis Visoiu Mistrih <francisvm at apple.com>
Subject: Re: [llvm-dev] RFC: Extending optimization reporting

On May 8, 2019, at 11:11 AM, Kaylor, Andrew <andrew.kaylor at intel.com<mailto:andrew.kaylor at intel.com>> wrote:

Hi Adam,

Thanks for your input.

If I understand correctly, you’re saying that we can handle the loop versioning issue by explicitly identifying new loops as they are created. So, the unswitching optimization, for example, would report that it unswitched loop-0 at source location X, creating loop-1 and loop-2, and then later the vectorizer would report that it was unable to vectorize loop-1 at source location X, and later still that it was able to vectorize loop-2 at source location X. And at each stage the optimization pass emitting the remark would get the version information from loop metadata. Is that right?


If so, I think that is in basic agreement with what I wanted to do. I’ll need to talk about it with some of my co-workers who have been thinking about ways to solve this problem, but I like this direction.

As you say, unrolled loops are still a problem. There’s basically no place to reliably hang metadata so that we can report that an optimization is working with an unrolled portion of a loop.

I’m not sure about the inlining issue. At first glance it does sound as simple as you suggest. However, my colleagues who have done a lot of work with inlining tell me there are some complications that make recovering all of the necessary data to form a coherent description of what has happened difficult. They’ve explained it to me a couple of times, but I haven’t internalized the subtleties yet. I’ll have to get back to you about this part.

Please keep me posted about this.



From: anemet at apple.com<mailto:anemet at apple.com> <anemet at apple.com<mailto:anemet at apple.com>>
Sent: Friday, May 03, 2019 2:27 PM
To: Kaylor, Andrew <andrew.kaylor at intel.com<mailto:andrew.kaylor at intel.com>>
Cc: llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>; Francis Visoiu Mistrih <francisvm at apple.com<mailto:francisvm at apple.com>>
Subject: Re: [llvm-dev] RFC: Extending optimization reporting

Hi Andrew,

On Apr 30, 2019, at 2:47 PM, Kaylor, Andrew via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

I would like to begin a discussion about updating LLVM's opt-report infrastructure. There are some things I'd like to be able to do with optimization reports that I don't think can be done, or at least aren't natural to do, with the current implementation.

I understand that there is a lot of code in place already to produce optimization remarks, and one of my explicit goals is to minimize the amount of updating existing code while still enabling the new features I would like to support. I have some ideas in mind for how to achieve what I'm proposing, but I want to start out by just describing the desired results and if we can reach a consensus on these being nice things to have then we can move on to talking about the best way to get there.

I think the extensions I have in mind can broadly be organized into two categories: (1) ways to support different sorts of output, and

As you may have seen Francis’ recent work, we have some plans in this area as well.  Francis is going to cover that part ...

(2) ways to create connections between different events represented in the report.

… let me cover this part.

Near as I can tell, the only support we have in the code base is for YAML output. I think I could implement a new RemarkStreamer to get other formats, but nothing in the LLVM code base does that. Is that correct?

I'd like to be able to:
- Embed some subset of optimizations remarks as annotations in the generated assembly output
- Embed the remarks in the generated executable in a binary format consumed by the Intel Advisor tool
- Produce text output in a format recognized by Microsoft Visual Studio or other IDEs

The last of these is probably straightfoward since it's basically a streaming format such as the current infrastructure expects. The other two seem like they might be more complicated, since they involve keeping the information around, potentially across LTO, and correlated with the evolving IR until the final machine code or assembly is produced.

This leads me back to my second category of extensions, creating connections between different events in the opt report. My goal here is to be able to produce some kind of coherent report after compilation is complete that lets the user make some sense of how the IR evolved over the course of compilation and what effects that may have had on optimizations. This mostly has to do with the handling of loops, vectorization, and inlining.

Let's say, for example, I've got code like this:

for (...)
    if (lic)

And the loop-unswitch pass turns it into this:

if (lic)
    for (...)
        A; B; C
    for (...)
        A; C

Now let's say the vectorizer for some reason is able to vectorize the loop in the else-clause but not the if-clause. (I don't know if this kind of thing is possible with the current phase ordering, but I think this theoretical example illustrates the idea anyway.)

I want some way to produce a report that tells the user about the existence of the two loops that were created when we unswitched the loop so that we can then tell the user in some sensible way that we couldn't vectorize one loop but that we could vectorize the second.

I'm not sure what the opt-viewer would currently do with a case like this, but what I want to avoid is getting stuck where the report we can emit essentially conveys the following not very helpful information.

for (...)
  // Loop was unswitched.
  // Loop could not be vectorized because...
  // Loop was vectorized.
    if (lic)

Instead I'd like to have a way to produce something like this:

for (...)
  // Loop was unswitched for condition (srcloc)
  //   Unswitched loop version #1
  //     Unswitched for IF condition (srcloc)
  //     Loop was not vectorized:
  //   Unswitched loop version #2
  //     Loop was vectorized

The primary thing missing, I think, is a way for the vectorizer to give some indication of which version of the loop it is talking about in its optimization remarks and maybe a way for the opt-viewer to be able to make sense of that.

Likewise, there are things I want to be able to track with inlining. Let's say we go through the inlining pass pre-LTO and we make some decisions and report them. Then during LTO we go through another round of inlining and possibly make different decisions. I'd like to be able to either produce a report that shows just the inlining from the LTO pass or produce a report that shows a composite of all the inlining decisions that we made.

I think the general problem is that we’re currently missing the ability to distinguish between code versions generated for the same source code, i.e. as functions get inlined multiple times or as loops get versioned.  (There is also loop-unrolling where we duplicate iterations with potentially removing the loop altogether.)

I think that for the inlined case we can walk the inlinedAt metadata to identify the code version in question.  For loops we could attach a version number to the loop id.  Then as long as the versioning transformation properly declares the new version, subsequent transformations can refer to code version they modify.  Then on the client side we should have all the information to reconstruct what happened.

I see this as essentially extending the DebugLoc field with code version information in the remark.

This still wouldn’t handle unrolling which would require some new metadata but it would be a good start.

What do you think?


We tried something like this with an inlining report before (https://reviews.llvm.org/D19397), but it had the misfortune of being proposed at about the same time that the current opt-viewer mechanism was being developed and we didn’t manage to get aligned with that. I’m hoping that we can correct that now.
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190523/31f63eed/attachment.html>

More information about the llvm-dev mailing list