[clang] [clang][Diagnostics] Highlight code snippets (PR #66514)

[Aaron Ballman via cfe-commits]

Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>,
Timm =?utf-8?q?Bäder?= <tbaeder at redhat.com>
Message-ID:
In-Reply-To: <llvm.org/llvm/llvm-project/pull/66514 at github.com>


AaronBallman wrote:

> So, I have a test file of about 4.5MB where every line is a `#warning`. Not sure how I should create a "similar" file without diagnostics though.
> 
> ```
> # ls -lah test.cpp
> -rw-r--r--. 1 root root 4.5M Dec 12 02:52 test.cpp
> # wc -l test.cpp
> 357391 test.cpp
> ```
> 
> With a release build, the time is:
> 
> ```
> # time bin/clang++ -cc1 ./test.cpp > out.txt 2>&1
> real    1m3.967s
> user    0m52.142s
> sys     0m11.606s
> ```
> 
> this is _without_ any highlighting, since the file is too big.
> 
> Here's the same file with highlighting:
> 
> ```
> # time bin/clang++ -cc1 ./test.cpp -fmax-highlight-file-size=5242880 > out.txt 2>&1
> real    72m28.551s
> user    72m11.610s
> sys     0m11.561s
> ```
> 
> (not that we definitely shouldn't take any longer here, since we don't emit any highlighting anyway because we're piping into a file. Need to fix that).

It seems to me that we either pay a small cost (~.10% or less) for checkpoints or we make people who have a lot of diagnostics pay a huge cost (going from 1.5 minutes to over an hour) for relexing, and that makes me think we should go with checkpoints instead of relexing the whole file. I realize this was a pathological test case and shouldn't be representative of real world code, but there are plenty of projects out there with *tons* of warnings being emitted and I don't think we can cause that much of a slowdown for their compile times. WDYT?

https://github.com/llvm/llvm-project/pull/66514