[llvm] update_test_checks: keep meta variables stable by default (PR #76748)

[Jannik Silvanus via llvm-commits]

Nicolai =?utf-8?q?Hähnle?= <nicolai.haehnle at amd.com>,
Nicolai =?utf-8?q?Hähnle?= <nicolai.haehnle at amd.com>,
Nicolai =?utf-8?q?Hähnle?= <nicolai.haehnle at amd.com>,
Nicolai =?utf-8?q?Hähnle?= <nicolai.haehnle at amd.com>
Message-ID:
In-Reply-To: <llvm.org/llvm/llvm-project/pull/76748 at github.com>


https://github.com/jasilvanus commented:

Thanks, this is very useful. I often resort to custom diff tools trying to filter out test churn caused by renamed variables.

I think we should add more comments explaining how the algorithm works and why we are doing what we are doing (see my inline comments in that regard).

Could you please add a more complicated test that tests the recursion and crossing-free matching part? 
I suspect the existing test case is trivially resolved in the top level without any of this.

How strongly do you feel about enabling this by default (now)? How should this feature (and resetting of names) be integrated into development work flows?

I'm a bit worried about defaulting to behavior where checked-in test lines depends on history, but on the other hand not making this the default will mean it's often not used where applicable (or reviewers have to request it), and I'd expect most UTC uses to prefer names to be perserved, except for dedicated test normalization changes.

Independently, we may want to test this a bit more before enabling by default, but that is only a temporary consideration.

https://github.com/llvm/llvm-project/pull/76748