[cfe-dev] [RFC] Add include-what-you-use tool to clang-tools-extra

Thu Sep 7 09:03:21 PDT 2017

Reimplementing the tool using the tooling available in the LLVM project would seem more appropriate to an LLVM-project tool. ☺

I am not really familiar with the original IWYU tool, but one thing I remember from James' work is that it would be fairly easy to implement different policies.  For example, minimizing the number of #includes, versus always directly including the header that declares everything actually used in the source.  That kind of flexibility is great.
--paulr

From: cfe-dev [mailto:cfe-dev-bounces at lists.llvm.org] On Behalf Of Manuel Klimek via cfe-dev
Sent: Thursday, September 07, 2017 4:47 AM
To: jh7370.2008 at my.bristol.ac.uk; Benjamin Kramer
Cc: Richard Smith; Clang Dev
Subject: Re: [cfe-dev] [RFC] Add include-what-you-use tool to clang-tools-extra

On Thu, Sep 7, 2017 at 12:25 PM James Henderson <jh7370.2008 at my.bristol.ac.uk<mailto:jh7370.2008 at my.bristol.ac.uk>> wrote:
In principle, I'd support having an IWYU tool in clang-extra-tools - it's something that we'd have found useful in the past in work I and the team I work with have done.
I did a quick prototype several months ago of an IWYU-style tool, based on the clang libraries (particularly the AST_MATCHER, PPCallbacks and other related code), and without looking directly at the existing implementation particularly. I was able to get it to do the basic stuff fairly quickly (only a few days' work), although it certainly wasn't as advanced as the actual IWYU. It was an interesting experiment however.
Without looking into all the details of how the existing IWYU is implemented, and based on my own experiences and what Manuel said earlier, I wonder whether it might be worth building up an IWYU tool that is inspired by, but not necessarily identical to, the existing one?

Yea. One question is how much of the "use" part of iwyu that's already encoded in clang as part of modules we could re-use. The whole thing might be a lot simpler if we put in an interface to Sema to get at the full information while parsing, and implementing iwyu as "as-if" warnings.
Ben did explore this for a bit, but I don't think it went anywhere.

Regards,
James

On 5 September 2017 at 13:21, Benjamin Kramer via cfe-dev <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>> wrote:
include-fixer (sorry for the name) solves the problem of adding missing includes. IWYU solves the problem of removing unused includes and resolving transitive includes. There is almost zero overlap between those two tools.

On Tue, Sep 5, 2017 at 2:19 PM Gábor Horváth <xazax.hun at gmail.com<mailto:xazax.hun at gmail.com>> wrote:
Hi!
Maybe I am missing something but could you summarize what is the difference between the iwyu tool and the already existing include-fixer in the repository?
Would iwyu deprecate include-fixer? Or does it make more sense to improve include-fixer and deprecate iwyu? Or are they completely separate?
Thanks,
Gábor

On 5 September 2017 at 09:42, Manuel Klimek via cfe-dev <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>> wrote:
I'm not fundamentally opposed to the idea, but I'd expect it to be rather painful - speaking from experience of having upstreamed a non-trivially sized chunk of google-style code into clang, which was still *significantly* smaller than iwyu.

Generally, we need patches to be:
1. LLVM style
2. incremental, with design ideas discussed / vetted in review

(1) is just a big chunk of work, but (2) can quickly lead code into a very different direction from where it is now; that said, I do believe that it would make the code, especially for iwyu, a lot better - but it would also make it an incredible amount of work, knowing how much time went into creating iwyu in the first place.

Additionally, with C++ modules, a new idea of "use" is emerging (I don't know whether standardization is far enough for that to be reliable), and I think we could / should and will build an "iwyu" implementation on top of that.
That doesn't make a non-modules iwyu useless, but I'd argue that if we want iwyu to live within clang-tools-extra, we want that to be aligned with the semantics of "modules-use", which I believe to be somewhat different; again, nothing is a show-stopper here, but I'd expect a significant amount of work.

Finally, given the current rate of tooling contributions across clang-tidy / clang-format / tooling / refactoring, and the number of upstream reviewers we have, I'd additionally expect the process to be rather slow; for example, the refactoring contributions have a much higher priority currently, and even those often take (imo) too long to be reviewed (partially my fault :)

With all that said, I don't want to discourage you from trying, but I want to set clear expectations - it might feel / look like a rewrite of iwyu from scratch.

Cheers,
/Manuel

On Tue, Sep 5, 2017 at 3:20 AM Volodymyr Sapsai via cfe-dev <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>> wrote:
As another include-what-you-use maintainer I support the proposal. I am doing so solely in my personal capacity and am not representing any third parties.

Regards,
Volodymyr
On Mon, Sep 4, 2017 at 13:41 Kim Gräsman via cfe-dev <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>> wrote:
Hi all,

This is a proposal to integrate include-what-you-use [1] into clang-tools-extra.

# Background
The include-what-you-use tool analyzes #includes in C and C++ files and
recommends how to improve them. The goal is to capture symbol dependencies in
code and produce the minimal set of #includes to satisfy these symbol
dependencies. For more information you can check the project site [2], docs, and
presentation from 2010 LLVM Developers' Meeting [3].

# Benefits
Migration to clang-tools-extra doesn't come without a cost, so I want to list
some of the benefits this move yields.

## For Clang community
* Ability to reuse some of IWYU analysis for other purposes. Currently IWYU is
  distributed as a CLI tool and has no API but it is possible to split out a
  separate library. I think there could be some integration potential with the
  budding refactoring tools, for example.
* More community input in deciding further IWYU direction to help it to be more
  useful for various parties.

## For include-what-you-use users
* Easier distribution and use. For users already using other Clang tools it
  should be easier to use IWYU as any other Clang tool. I also expect it to make
  life easier for people packaging include-what-you-use for various *nix
  distributions.
* Moving the tool towards consistency with other Clang tools.

## For include-what-you-use project
* Exposure to more users.
* Easier release process. Instead of releasing IWYU separately, it could be
  bundled with LLVM+Clang releases. It shouldn't incur more work for LLVM+Clang
  releases as the main complexity comes from tracking different branches and
  building binaries for different platforms.
* More resiliency as the project becomes more community-owned instead of
  personally-owned.

# Potential downsides
When new Clang sub-projects are proposed, one of the most common concerns is the
maintenance burden. Dumping the code and walking away to let the community
support the code is unacceptable. The longevity of the project demonstrates
commitment to maintaining the project. The history on GitHub shows that
include-what-you-use is not a passing fancy that will be discarded and forgotten
in a week or two.

What is your opinion, is there value in having include-what-you-use in
clang-tools-extra?

Thanks for any input,
- Kim

[1] https://github.com/include-what-you-use/include-what-you-use
[2] https://include-what-you-use.org/
[3] http://llvm.org/devmtg/2010-11/
_______________________________________________
cfe-dev mailing list
cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
_______________________________________________
cfe-dev mailing list
cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

_______________________________________________
cfe-dev mailing list
cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170907/873f100d/attachment.html>