[PATCH] D98278: [test] Add ability to get error messages from CMake for errc substitution

[Markus Böck via Phabricator via llvm-commits]

zero9178 added a comment.

In D98278#2638005 <https://reviews.llvm.org/D98278#2638005>, @mstorsjo wrote:

> In D98278#2637866 <https://reviews.llvm.org/D98278#2637866>, @zero9178 wrote:
>
>> In D98278#2637826 <https://reviews.llvm.org/D98278#2637826>, @mstorsjo wrote:
>>
>>> Btw, while this change does explain _what_ it does, it doesn't actually say the exact reason _why_. Cleanliness? Sure, that's nice... Or is it a case where e.g. some translations produce different error messages?
>>
>> Now that you mention it, it's indeed not as clear as I thought. But yes, in the case of MSVCs STL, the messages from `std::error_codes` which are used by various LLVM tools produce different strings then using `strerror` (the C function also called by Python) with the same error codes (Specifically, it has different casing).
>
> Ok, but would e.g. a case insensitive comparison have worked instead of this?
>
> And didn't the python script have hardcoded strings, specifically for the MSVC case? Why weren't they written with the right casing for the case that they're supposed to match? I.e. was it an issue with the existing hardcoded strings, or did they work in one case but not another one?

Changes in this patch are based on this one https://reviews.llvm.org/D97472. In the discussion there it was deemed not a good solution to use case insensitive comparison as that would make any other matches case insensitive as well, which might be a source of bugs.

In D98278#2638023 <https://reviews.llvm.org/D98278#2638023>, @mstorsjo wrote:

> In D98278#2637866 <https://reviews.llvm.org/D98278#2637866>, @zero9178 wrote:
>
>> In D98278#2637826 <https://reviews.llvm.org/D98278#2637826>, @mstorsjo wrote:
>>
>>> Btw, while this change does explain _what_ it does, it doesn't actually say the exact reason _why_. Cleanliness? Sure, that's nice... Or is it a case where e.g. some translations produce different error messages?
>>
>> Now that you mention it, it's indeed not as clear as I thought. But yes, in the case of MSVCs STL, the messages from `std::error_codes` which are used by various LLVM tools produce different strings then using `strerror` (the C function also called by Python) with the same error codes (Specifically, it has different casing).
>
> Or turning the question another way around: We have a couple different bots that build and run the tests, successfully, with MSVC configurations. Are there tests that failed for you in your configuration, that succeed in the setup of the bots? Or are there other tests that aren't run as part of bots that you're fixing? It's all still very vague to me.

Prior to this patch it worke with an MSVC configuration already as the strings were correctly hardcoded for the MSVC STL. Problem was when using any other compiler configuration on Windows. In my case I am using your llvm-mingw distribution to build all of LLVM and since it uses libc++ it produces different error messages from the one in MSVC STL. I observed the same problem when using GCC on Windows and if one were to theoretically use libc++ with clang-cl in a MSVC environment, tests would be failing as well due to a mismatch in error strings.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D98278/new/

https://reviews.llvm.org/D98278