[PATCH] D54141: [clang-tidy] add deduplication support for run-clang-tidy.py

Jonas Toth via Phabricator via cfe-commits cfe-commits at lists.llvm.org
Tue Nov 6 08:35:38 PST 2018

JonasToth added a comment.

Thank you for the comment!

In https://reviews.llvm.org/D54141#1288809, @steveire wrote:

> This feature seems like a good idea. I started writing it too some months ago, but then I changed tactic and worked on distributing the refactor over the network instead. As far as I know, your deduplication would not work with a distributed environment.

I agree that it would probably not work. It might enable a two-stage deduplication, but I don't know if that would be viable.

> However, it seems that both features can exist.
> You use a regex to parse the clang output. Why not use the already-machine-readable yaml output and de-duplicate based on that? I think the design would be something like:
> - Run clang-tidy in a quiet mode which only exports yaml and does not issue diagnostics
> - Read the yaml in your python script
> - Add the entries to your already-seen cache
> - For any entry which was not already there
>   - Write the entries to a new yaml file
>   - Use clang-apply-replacements --issue-diags the_new_file.yaml to actually cause the new diagnostics to be issued (they were omitted from the clang-tidy run).
>     This avoids fragile parsing of the output from clang, instead relying on the machine-readable format.

In principle this approach seems more robust and I am not claiming my approach is robust at all :)
The point hokein raised should be considered first in my opinion. If clang-tidy itself is already parallel we should definitely deduplicate there. This is something I would put more
effort in. The proposed solution is more a hack to get my buildbot running and find transformation bugs and provide real-world data for checks we implement. :)

> I think clang-apply-replacements already does de-duplication, so it's possible that could take more responsibility.

Yes, the emitted fixes are deduplicated but i think we need something even if no fixes are involved.

> Also, I think your test content is too big. I suggest trying to write more contained tests for this.

I wanted to have a mix of both real snippets and some unit-tests on short examples. Do you think its enough if i shorten the list of fields that the CSA output contains for the padding checker?

  rCTE Clang Tools Extra


More information about the cfe-commits mailing list