[cfe-dev] [RFC] Add include-what-you-use tool to clang-tools-extra

Thu Sep 7 03:25:43 PDT 2017

In principle, I'd support having an IWYU tool in clang-extra-tools - it's
something that we'd have found useful in the past in work I and the team I
work with have done.

I did a quick prototype several months ago of an IWYU-style tool, based on
the clang libraries (particularly the AST_MATCHER, PPCallbacks and other
related code), and without looking directly at the existing implementation
particularly. I was able to get it to do the basic stuff fairly quickly
(only a few days' work), although it certainly wasn't as advanced as the
actual IWYU. It was an interesting experiment however.

Without looking into all the details of how the existing IWYU is
implemented, and based on my own experiences and what Manuel said earlier,
I wonder whether it might be worth building up an IWYU tool that is
inspired by, but not necessarily identical to, the existing one?

Regards,
James

On 5 September 2017 at 13:21, Benjamin Kramer via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> include-fixer (sorry for the name) solves the problem of adding missing
> includes. IWYU solves the problem of removing unused includes and resolving
> transitive includes. There is almost zero overlap between those two tools.
>
> On Tue, Sep 5, 2017 at 2:19 PM Gábor Horváth <xazax.hun at gmail.com> wrote:
>
>> Hi!
>>
>> Maybe I am missing something but could you summarize what is the
>> difference between the iwyu tool and the already existing include-fixer in
>> the repository?
>> Would iwyu deprecate include-fixer? Or does it make more sense to improve
>> include-fixer and deprecate iwyu? Or are they completely separate?
>>
>> Thanks,
>> Gábor
>>
>> On 5 September 2017 at 09:42, Manuel Klimek via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> I'm not fundamentally opposed to the idea, but I'd expect it to be
>>> rather painful - speaking from experience of having upstreamed a
>>> non-trivially sized chunk of google-style code into clang, which was still
>>> *significantly* smaller than iwyu.
>>>
>>> Generally, we need patches to be:
>>> 1. LLVM style
>>> 2. incremental, with design ideas discussed / vetted in review
>>>
>>> (1) is just a big chunk of work, but (2) can quickly lead code into a
>>> very different direction from where it is now; that said, I do believe that
>>> it would make the code, especially for iwyu, a lot better - but it would
>>> also make it an incredible amount of work, knowing how much time went into
>>> creating iwyu in the first place.
>>>
>>> Additionally, with C++ modules, a new idea of "use" is emerging (I don't
>>> know whether standardization is far enough for that to be reliable), and I
>>> think we could / should and will build an "iwyu" implementation on top of
>>> that.
>>> That doesn't make a non-modules iwyu useless, but I'd argue that if we
>>> want iwyu to live within clang-tools-extra, we want that to be aligned with
>>> the semantics of "modules-use", which I believe to be somewhat different;
>>> again, nothing is a show-stopper here, but I'd expect a significant amount
>>> of work.
>>>
>>> Finally, given the current rate of tooling contributions across
>>> clang-tidy / clang-format / tooling / refactoring, and the number of
>>> upstream reviewers we have, I'd additionally expect the process to be
>>> rather slow; for example, the refactoring contributions have a much higher
>>> priority currently, and even those often take (imo) too long to be reviewed
>>> (partially my fault :)
>>>
>>> With all that said, I don't want to discourage you from trying, but I
>>> want to set clear expectations - it might feel / look like a rewrite of
>>> iwyu from scratch.
>>>
>>> Cheers,
>>> /Manuel
>>>
>>> On Tue, Sep 5, 2017 at 3:20 AM Volodymyr Sapsai via cfe-dev <
>>> cfe-dev at lists.llvm.org> wrote:
>>>
>>>> As another include-what-you-use maintainer I support the proposal. I am
>>>> doing so solely in my personal capacity and am not representing any third
>>>> parties.
>>>>
>>>> Regards,
>>>> Volodymyr
>>>> On Mon, Sep 4, 2017 at 13:41 Kim Gräsman via cfe-dev <
>>>> cfe-dev at lists.llvm.org> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> This is a proposal to integrate include-what-you-use [1] into
>>>>> clang-tools-extra.
>>>>>
>>>>> # Background
>>>>> The include-what-you-use tool analyzes #includes in C and C++ files and
>>>>> recommends how to improve them. The goal is to capture symbol
>>>>> dependencies in
>>>>> code and produce the minimal set of #includes to satisfy these symbol
>>>>> dependencies. For more information you can check the project site [2],
>>>>> docs, and
>>>>> presentation from 2010 LLVM Developers' Meeting [3].
>>>>>
>>>>> # Benefits
>>>>> Migration to clang-tools-extra doesn't come without a cost, so I want
>>>>> to list
>>>>> some of the benefits this move yields.
>>>>>
>>>>> ## For Clang community
>>>>> * Ability to reuse some of IWYU analysis for other purposes. Currently
>>>>> IWYU is
>>>>>   distributed as a CLI tool and has no API but it is possible to split
>>>>> out a
>>>>>   separate library. I think there could be some integration potential
>>>>> with the
>>>>>   budding refactoring tools, for example.
>>>>> * More community input in deciding further IWYU direction to help it
>>>>> to be more
>>>>>   useful for various parties.
>>>>>
>>>>> ## For include-what-you-use users
>>>>> * Easier distribution and use. For users already using other Clang
>>>>> tools it
>>>>>   should be easier to use IWYU as any other Clang tool. I also expect
>>>>> it to make
>>>>>   life easier for people packaging include-what-you-use for various
>>>>> *nix
>>>>>   distributions.
>>>>> * Moving the tool towards consistency with other Clang tools.
>>>>>
>>>>> ## For include-what-you-use project
>>>>> * Exposure to more users.
>>>>> * Easier release process. Instead of releasing IWYU separately, it
>>>>> could be
>>>>>   bundled with LLVM+Clang releases. It shouldn't incur more work for
>>>>> LLVM+Clang
>>>>>   releases as the main complexity comes from tracking different
>>>>> branches and
>>>>>   building binaries for different platforms.
>>>>> * More resiliency as the project becomes more community-owned instead
>>>>> of
>>>>>   personally-owned.
>>>>>
>>>>> # Potential downsides
>>>>> When new Clang sub-projects are proposed, one of the most common
>>>>> concerns is the
>>>>> maintenance burden. Dumping the code and walking away to let the
>>>>> community
>>>>> support the code is unacceptable. The longevity of the project
>>>>> demonstrates
>>>>> commitment to maintaining the project. The history on GitHub shows that
>>>>> include-what-you-use is not a passing fancy that will be discarded and
>>>>> forgotten
>>>>> in a week or two.
>>>>>
>>>>> What is your opinion, is there value in having include-what-you-use in
>>>>> clang-tools-extra?
>>>>>
>>>>> Thanks for any input,
>>>>> - Kim
>>>>>
>>>>> [1] https://github.com/include-what-you-use/include-what-you-use
>>>>> [2] https://include-what-you-use.org/
>>>>> [3] http://llvm.org/devmtg/2010-11/
>>>>> _______________________________________________
>>>>> cfe-dev mailing list
>>>>> cfe-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>>>
>>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170907/cb4412c9/attachment.html>