<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div>On Oct 29, 2020, at 10:28 AM, Mircea Trofin via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""><blockquote type="cite" class=""><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Hello all,<br class=""></div><div class=""><br class=""></div><div class="">TL;DR; if you used FileCheck --check-prefixes and you missed (misspelled, for instance) one of the prefixes in your test, FileCheck silently ignores that and the test passes. </div><div class=""><br class=""></div><div class="">1579 tests have this property. </div></div></div></blockquote><div><br class=""></div><div>Wow, that is quite a “whoops”. Thank you for catching this, your plan sounds really great.<br class=""></div><div><br class=""></div><div>-Chris</div><div class=""><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class=""><b class=""><br class=""></b></div><div class=""><b class="">The details</b></div><div class=""><b class="">=========</b></div><div class="">Please refer to <a href="https://reviews.llvm.org/D90281" class="">https://reviews.llvm.org/D90281</a> and the discussion there for more details (make sure you open "older changes" for full context)</div><div class=""><br class=""></div><div class="">The problem is covered by the TL;DR;. </div><div class=""><br class=""></div><div class="">The proposal is to add an explicit flag to FileCheck, --allow-unused-prefixes, to indicate whether the current behavior is intended (for instance, jdoerfert contributed a scenario where that is the case). </div><div class=""><br class=""></div><div class="">We want the default behavior to be 'strict', i.e. --allow-unused-prefixes=false. Doing that right now would lead to 1500 test failures.</div><div class=""><br class=""></div><div class="">To get there (thanks, maskray, for suggestion), we propose we:</div><div class="">* land D90281 where the flag is introduced, but is flipped to match today's behavior</div><div class="">* employ a 'busy beavers' approach, where test maintainers patch their tests:</div><div class=""> - either leveraging the flag, to explicitly indicate that unused prefixes is intended (i.e. add --allow-unused-patches=true); or</div><div class=""> - fix the test (e.g. maybe there was a misspelling issue/omission/etc).</div><div class=""><br class=""></div><div class="">A spreadsheet with the failing tests is available <a href="https://docs.google.com/spreadsheets/d/1o6q3XH1n3DDyyccnYZ_kVfhFbTDzC_S09e973_cwYuw/edit?usp=sharing" class="">here</a> [1].</div><div class=""><br class=""></div><div class="">The request to the community members is to please sign up for their respective area in the spreadsheet, and then mark it completed when that's the case (yes/no in the respective column).</div><div class=""><br class=""></div><div class="">When all the tests are fixed, we will then flip --allow-unused-prefixes to false by default.</div><div class=""><br class=""></div><div class="">Meanwhile, please consider leveraging the flag explicitly when you author new tests that use --check-prefixes. That can be then cleaned up easily after we switch to the 'strict' behavior.</div><div class=""><br class=""></div><div class="">Thanks!</div><div class=""><br class=""></div><div class="">[1] <a href="https://docs.google.com/spreadsheets/d/1o6q3XH1n3DDyyccnYZ_kVfhFbTDzC_S09e973_cwYuw/edit?usp=sharing" class="">https://docs.google.com/spreadsheets/d/1o6q3XH1n3DDyyccnYZ_kVfhFbTDzC_S09e973_cwYuw/edit?usp=sharing</a></div></div>
_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<br class=""></div></blockquote></div><br class=""></body></html>