[PATCH] D90109: [clang-tidy] Use ANSI escape codes for --use-color on Windows

[David Sanders via Phabricator via llvm-commits]

dsanders11 added a comment.

Added a few inline comments for clarification.

**Note**: I was not able to get a debug build and unit tests working on my Windows set up, so this has only been tested manually. I'm not sure if the unit test added in D79477 <https://reviews.llvm.org/D79477> runs on Windows, but  in theory it should be broken on Windows since it expects ANSI escape codes in the output.



================
Comment at: clang-tools-extra/clang-tidy/ClangTidy.cpp:113
     DiagPrinter->BeginSourceFile(LangOpts);
+#if defined(_WIN32)
+    if (DiagOpts->ShowColors && !llvm::sys::Process::StandardOutIsDisplayed()) {
----------------
To prevent any unintended changes this has been confined to only Windows, but the code should be safe to run on any platform. I believe it would be a no-op on Unix, which always uses ANSI escape codes and UseANSIEscapeCodes is a no-op.


================
Comment at: clang-tools-extra/clang-tidy/ClangTidy.cpp:114
+#if defined(_WIN32)
+    if (DiagOpts->ShowColors && !llvm::sys::Process::StandardOutIsDisplayed()) {
+      llvm::sys::Process::UseANSIEscapeCodes(true);
----------------
Limited to only non-displayed standard out so that using '--use-color' from a console will be a no-op, rather than turning on ANSI escape codes and making it act differently than a default invocation which has color on by default.


================
Comment at: llvm/lib/Support/Windows/Process.inc:330
     DWORD Mode;
-    GetConsoleMode(Console, &Mode);
-    Mode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
-    SetConsoleMode(Console, Mode);
+    if (GetConsoleMode(Console, &Mode) != 0) {
+      Mode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
----------------
Fixes a crash when calling UseANSIEscapeCodes if standard out isn't a console.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D90109/new/

https://reviews.llvm.org/D90109