[LLVMdev] [RFC] Add warning capabilities in LLVM.

Sat Jul 20 12:28:18 PDT 2013

Hi Quentin,

I have a particular use case that I'd like this infrastructure to support, and I'm not sure if your proposal captures this or not: hints from the autovectorization passes (and other loop optimizations). Specifically, I'd like to be able to relay messages like this:

 - "*This loop* cannot be optimized because the induction variable *i* is unsigned, and cannot be proved not to wrap"

 - "*This loop* cannot be vectorized because the compiler cannot prove that memory read from *a* does not overlap with memory written to through *b*"

 - "*This loop* cannot be vectorized because the compiler cannot prove that it is unconditionally safe to dereference the pointer *a*.

and I'd like the frontend to get enough information through the interface to turn the starred items back into higher-level-language constructs (that way they can be turned into hyperlinks, underlined, or whatever). I think this might be possible by providing references to any associated debug metadata. Maybe we can define some simple markup, and have each marked item correspond to an entry in an array of metadata poitners? What do you think?

Thanks,
Hal

----- Original Message -----
> 
> 
> Hi,
> 
> 
> As no new suggestions came in, I would like to share my plans to move
> forward.
> 
> 
> — Feedbacks are very welcome! --
> 
> 
> Like I summed up in my previous email (see below), there are two
> orthogonal approaches to this problem.
> I have chosen the second one (extend the reporting already used for
> inline asm).
> 
> 
> Regarding the first approach (front-end querying the back-end to
> build up diagnostic), it needs further discussion if anyone wants to
> go in that direction. Although I think this is interesting, I leave
> that for someone else.
> 
> 
> 
> 
> ** Proposed Plan **
> 
> 
> Extend what already exists: back-end reports to the front-end through
> hooks.
> 
> 
> The detail plan is the following (still hand wavy, I did not look
> into the actual code base, just had some high level discussions):
> 1. LLVM: Extend the current diagnostic hook to pass:
> 1.1. A kind of diagnostic (InlineAsm, StackSize, Other <TBD>).
> 1.2. A boolean hinting whether the diagnostic is an error or a
> warning.
> 2. Clang: Map the diagnostic to the proper (pre-defined) group based
> on the passed kind.
> 
> 
> 
> 
> ** Remarks on **
> 
> 
> 1.1. This is supposed to change rarely, that is why a static approach
> should do the trick. The ‘Other’ kind represents a fall-back cases
> for thing we did not anticipate (do not get too hung up on the
> names, they have to be defined (name, which kind of back-end
> diagnostic we want to expose, etc.)).
> 1.2. Each front-end does not have clang capabilities, thus this
> boolean will help the "front-end" to figure out what it should do
> with that diagnostic (e.g., keep going or abort). Clang may ignore
> them.
> 
> 
> 2. Obviously, the kind of diagnostic has to be shared between Clang
> and LLVM, this is the awkward part, but again, this should rarely
> change.
> 
> 
> 
> 
> Thanks again for all the contributions made so far and for the future
> ones as well!
> 
> 
> Cheers,
> 
> 
> -Quentin
> 
> 
> 
> On Jul 17, 2013, at 11:34 AM, Quentin Colombet < qcolombet at apple.com
> > wrote:
> 
> 
> 
> Hi,
> 
> 
> Thanks all for the insight comments.
> 
> 
> Let me sum up at a high level what proposals we actually have (sorry
> if I misinterpreted or missed something, do not hesitate to correct
> me):
> 
> 
> 1. Make LLVM defines some APIs that exposes some internal information
> so that a front-end can use them to build-up some diagnostics.
> 
> 
> 2. Make LLVM builds up a diagnostic and let a front-end maps this
> diagnostic to the right warning/error group.
> 
> 
> 
> 
> In my opinion, both approaches are orthogonal and have different
> advantages based on what goal we are pursuing.
> 
> 
> To be more specific, with the first approach, front-end people can
> come up with new warnings/errors and diagnostics without having to
> modify the back-end. On the other hand, with the second approach,
> back-end people can emit new warnings/errors without having to
> modify the front-end (BTW, it would be nice if we come up at least
> in clang with a consistent way to pass down options for emitting
> back-end warning/error when applicable, i.e., without -mllvm but
> with -W<whatever>).
> 
> 
> What people are thinking?
> 
> 
> Thanks again for sharing your thoughts, I appreciate.
> 
> 
> Cheers,
> 
> 
> 
> -Quentin
> 
> 
> On Jul 17, 2013, at 9:38 AM, Evan Cheng < evan.cheng at apple.com >
> wrote:
> 
> 
> 
> 
> 
> 
> Sent from my iPad
> 
> On Jul 17, 2013, at 8:53 AM, Bob Wilson < bob.wilson at apple.com >
> wrote:
> 
> 
> 
> 
> 
> 
> On Jul 17, 2013, at 2:12 AM, Chandler Carruth < chandlerc at google.com
> > wrote:
> 
> 
> 
> On Tue, Jul 16, 2013 at 9:34 PM, Bob Wilson < bob.wilson at apple.com >
> wrote:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Jul 16, 2013, at 5:51 PM, Eli Friedman < eli.friedman at gmail.com >
> wrote:
> 
> 
> 
> 
> On Tue, Jul 16, 2013 at 5:21 PM, Quentin Colombet <
> qcolombet at apple.com > wrote:
> 
> 
> 
> ** Advices Needed **
> 
> 1. Decide whether or not we want such capabilities (if we do not we
> may just
> add sporadically the support for a new warning/group of
> warning/error).
> 2. Come up with a plan to implement that (assuming we want it).
> 
> The frontend should be presenting warnings, not the backend; adding a
> hook which provides the appropriate information shouldn't be too
> hard.
> Warnings coming out of the backend are very difficult to design well,
> so I don't expect we will add many. Also, keep in mind that the
> information coming out of the backend could be used in other ways; it
> might not make sense for the backend to decide that some piece of
> information should be presented as a warning. (Consider, for example,
> IDE integration to provide additional information about functions and
> loops on demand.)
> 
> I think we definitely need this. In fact, I tried adding something
> simple earlier this year but gave up when I realized that the task
> was bigger than I expected. We already have a hook for diagnostics
> that can be easily extended to handle warnings as well as errors
> (which is what I tried earlier), but the problem is that it is
> hardwired for inline assembly errors. To do this right, new warnings
> really need to be associated with warning groups so that can be
> controlled from the front-end.
> 
> 
> I agree with Eli that there probably won’t be too many of these.
> Adding a few new entries to clang’s diagnostic .td files would be
> fine, except that the backend doesn’t see those. It seems like we
> will need something in llvm that defines a set of “backend
> diagnostics”, along with a table in the frontend to correlate those
> with the corresponding clang diagnostics. That seems awkward at best
> but maybe it’s tolerable as long as there aren’t many of them.
> 
> 
> I actually think this is the wrong approach, and I don't think it's
> quite what Eli or I am suggestion (of course, Eli may want to
> clarify, I'm only really clarifying what *I'm* suggesting.
> 
> 
> I think all of the warnings should be in the frontend, using the
> standard and existing machinery for generating, controlling, and
> displaying a warning. We already know how to do that well. The
> difference is that these warnings will need to query the LLVM layer
> for detailed information through some defined API, and base the
> warning on this information. This accomplishes two things:
> 
> 
> 1) It ensures the warning machinery is simple, predictable, and
> integrates cleanly with everything else in Clang. It does so in the
> best way by simply being the existing machinery.
> 
> 
> 2) It forces us to design reasonable APIs in LLVM to expose to a FE
> for this information. A consequence of this will be to sort out the
> layering issues, etc. Another consequence will be a strong chance of
> finding general purpose APIs in LLVM that can serve many purposes,
> not just a warning. Consider JITs and other systems that might
> benefit from having good APIs for querying the size and makeup (at a
> high level) of a generated function.
> 
> 
> A nice side-effect is that it simplifies the complexity involved for
> simple warnings -- now it merely is the complexity of exposing the
> commensurately simple API in LLVM. If instead we go the route of
> threading a FE interface for *reporting* warnings into LLVM, we have
> to thread an interface with sufficient power to express many
> different concepts.
> 
> I don't understand what you are proposing.
> 
> 
> First, let me try to clarify my proposal, in case there was any
> confusion about that. LLVMContext already has a hook for
> diagnostics, setInlineAsmDiagnosticHandler() et al. I was suggesting
> that we rename those interfaces to be more generic, add a simple
> enumeration of whatever diagnostics can be produced from the
> backend, and add support in clang for mapping those enumeration
> values to the corresponding clang diagnostics. This would be a small
> amount of work and would also be consistent with everything you
> wrote above about reusing the standard and existing machinery for
> diagnostics in clang. For the record, I had started down that path
> in svn commits 171041 and 171047, but I reverted those changes in
> 174748 and 174860, since they didn't go far enough to make it work
> properly and it wasn't clear at the time whether we really needed
> it.
> 
> 
> Now let me try to understand what you're suggesting…. You refer
> several times to having clang query the LLVM layer. Is this to
> determine whether to emit a diagnostic for some condition? How would
> this work? Would you have clang insert extra passes to check for
> various conditions that might require diagnostics? I don't see how
> else you would do it, since clang's interface to the backend just
> sets up the PerFunctionPasses, PerModulePasses and CodeGenPasses
> pass managers and then runs them. Assuming you did add some special
> passes to check for problems, wouldn't those passes have to
> duplicate a lot of effort in some cases to find the answers? Take
> for example the existing warnings in
> IntrinsicLowering::LowerIntrinsicCall. Those badly need to be
> cleaned up. Would clang run a special pass to check for intrinsics
> that are not supported by the target? That pass would need to be
> implemented as part of clang so that it would have access to clang's
> diagnostic machinery, but it would also need to know details about
> what intrinsics are supported by the target. Interesting layering
> problems there…. Apologies if I'm misinterpreting your proposal.
> 
> We can't assume clang is the frontend or design a system that only
> works with clang. There are many systems that use llvm which are not
> even c compilers.
> 
> 
> Evan
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory