[cfe-dev] [RFC] Add include-what-you-use tool to clang-tools-extra

[Kim Gräsman via cfe-dev]

Hi Manuel,

Thanks for your thoughtful reply.

I'm not so worried about LLVM style, I think we can get there without too much
effort with a combination of common sense, sed and clang-format.

I'm a strong proponent of incremental patch-by-patch development, and would
absolutely prefer further IWYU development in that style. But I'm not sure I see
the point of re-implementing IWYU for the purpose of small, incremental
patches. I agree that many design choices would be different if built from
scratch, but it seems like a waste to throw away all the hard work that has
already gone into IWYU.

It would basically stop the project dead in its tracks; bug fixes, improvements
and contributions would need to be put on hold until a rewrite is
completed.

My preference would be to make incremental changes on top of the existing
functional code base, out-of-tree or in-tree, to make it more in line with LLVM
standards.

Can we go back to first principles and talk about the value gained by small
increments? Maybe there's a less disruptive way to reach the same goal.

Thanks,
- Kim

On Tue, Sep 5, 2017 at 9:42 AM, Manuel Klimek <klimek at google.com> wrote:
> I'm not fundamentally opposed to the idea, but I'd expect it to be rather
> painful - speaking from experience of having upstreamed a non-trivially
> sized chunk of google-style code into clang, which was still *significantly*
> smaller than iwyu.
>
> Generally, we need patches to be:
> 1. LLVM style
> 2. incremental, with design ideas discussed / vetted in review
>
> (1) is just a big chunk of work, but (2) can quickly lead code into a very
> different direction from where it is now; that said, I do believe that it
> would make the code, especially for iwyu, a lot better - but it would also
> make it an incredible amount of work, knowing how much time went into
> creating iwyu in the first place.
>
> Additionally, with C++ modules, a new idea of "use" is emerging (I don't
> know whether standardization is far enough for that to be reliable), and I
> think we could / should and will build an "iwyu" implementation on top of
> that.
> That doesn't make a non-modules iwyu useless, but I'd argue that if we want
> iwyu to live within clang-tools-extra, we want that to be aligned with the
> semantics of "modules-use", which I believe to be somewhat different; again,
> nothing is a show-stopper here, but I'd expect a significant amount of work.
>
> Finally, given the current rate of tooling contributions across clang-tidy /
> clang-format / tooling / refactoring, and the number of upstream reviewers
> we have, I'd additionally expect the process to be rather slow; for example,
> the refactoring contributions have a much higher priority currently, and
> even those often take (imo) too long to be reviewed (partially my fault :)
>
> With all that said, I don't want to discourage you from trying, but I want
> to set clear expectations - it might feel / look like a rewrite of iwyu from
> scratch.
>
> Cheers,
> /Manuel
>
> On Tue, Sep 5, 2017 at 3:20 AM Volodymyr Sapsai via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
>>
>> As another include-what-you-use maintainer I support the proposal. I am
>> doing so solely in my personal capacity and am not representing any third
>> parties.
>>
>> Regards,
>> Volodymyr
>> On Mon, Sep 4, 2017 at 13:41 Kim Gräsman via cfe-dev
>> <cfe-dev at lists.llvm.org> wrote:
>>>
>>> Hi all,
>>>
>>> This is a proposal to integrate include-what-you-use [1] into
>>> clang-tools-extra.
>>>
>>> # Background
>>> The include-what-you-use tool analyzes #includes in C and C++ files and
>>> recommends how to improve them. The goal is to capture symbol
>>> dependencies in
>>> code and produce the minimal set of #includes to satisfy these symbol
>>> dependencies. For more information you can check the project site [2],
>>> docs, and
>>> presentation from 2010 LLVM Developers' Meeting [3].
>>>
>>> # Benefits
>>> Migration to clang-tools-extra doesn't come without a cost, so I want to
>>> list
>>> some of the benefits this move yields.
>>>
>>> ## For Clang community
>>> * Ability to reuse some of IWYU analysis for other purposes. Currently
>>> IWYU is
>>>   distributed as a CLI tool and has no API but it is possible to split
>>> out a
>>>   separate library. I think there could be some integration potential
>>> with the
>>>   budding refactoring tools, for example.
>>> * More community input in deciding further IWYU direction to help it to
>>> be more
>>>   useful for various parties.
>>>
>>> ## For include-what-you-use users
>>> * Easier distribution and use. For users already using other Clang tools
>>> it
>>>   should be easier to use IWYU as any other Clang tool. I also expect it
>>> to make
>>>   life easier for people packaging include-what-you-use for various *nix
>>>   distributions.
>>> * Moving the tool towards consistency with other Clang tools.
>>>
>>> ## For include-what-you-use project
>>> * Exposure to more users.
>>> * Easier release process. Instead of releasing IWYU separately, it could
>>> be
>>>   bundled with LLVM+Clang releases. It shouldn't incur more work for
>>> LLVM+Clang
>>>   releases as the main complexity comes from tracking different branches
>>> and
>>>   building binaries for different platforms.
>>> * More resiliency as the project becomes more community-owned instead of
>>>   personally-owned.
>>>
>>> # Potential downsides
>>> When new Clang sub-projects are proposed, one of the most common concerns
>>> is the
>>> maintenance burden. Dumping the code and walking away to let the
>>> community
>>> support the code is unacceptable. The longevity of the project
>>> demonstrates
>>> commitment to maintaining the project. The history on GitHub shows that
>>> include-what-you-use is not a passing fancy that will be discarded and
>>> forgotten
>>> in a week or two.
>>>
>>> What is your opinion, is there value in having include-what-you-use in
>>> clang-tools-extra?
>>>
>>> Thanks for any input,
>>> - Kim
>>>
>>> [1] https://github.com/include-what-you-use/include-what-you-use
>>> [2] https://include-what-you-use.org/
>>> [3] http://llvm.org/devmtg/2010-11/
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev