[PATCH] D20401: [Lexer] Don't merge macro args from different macro files

Thu Sep 29 11:52:10 PDT 2022

sammccall added a comment.

Thanks Nick for the info! No kernel experience here, so if you have any particular suggestions about how to measure the workload you care about it'd be much appreciated (e.g. are particular files that are slow enough to measure in isolation, or is it better to do a full build)

In D20401#3824569 <https://reviews.llvm.org/D20401#3824569>, @nickdesaulniers wrote:

> But I wouldn't be surprised if other Linux distro's like RHEL bootstrap their clang distribution via GCC.  @tstellar or @serge-sans-paille or @nikic might know.  We did get a curious comment from a kernel developer recently claiming that clang was "twice as slow as GCC" which didn't make much sense; not sure if it was an exaggeration vs. precise measurement, but I wouldn't be surprised if evaluation order you identified plays into this, making the worst method even slower.  I'll try to find a link to the thread...

I'm sure this is common enough to be worth fixing if it's a real effect which I need to confirm. (MSVC is right-to-left too).

> Meanwhile, I think besides evaluating the high level logic in in `TokenLexer` and how it might be improved, I think there's potentially an opportunity for a "AOS vs. SOA" speedup in `SourceManager`.  `SourceManager::LoadedSLocEntryTable` is a `llvm::SmallVector<SrcMgr::SLocEntry>`. `SourceManager::getFileIDLoaded` only really cares about the `SLocEntry`'s `Offset`. I suspect we could get major memory locality wins by packing those into a standalone vector so that we could search them faster.

Ah, great point. SLocEntry is 24 bytes while Offset is only 4. SLocEntry is an important public API, but Offset is ~only used in SourceManager, so that refactoring might be doable. I guess we can cheaply prototype by redundantly storing offset *both* in a separate array used for search and in the SLocEntry.

>> @hokein or I will try to find time to take a stab at this.
>
> Awesome, please keep me in the loop.

Will do!

Repository:
  rL LLVM

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D20401/new/

https://reviews.llvm.org/D20401