[clang] [llvm] [clang][Modules][Serialization] Avoid serializing duplicate source location entries for shared input files (PR #209795)

[Ayokunle Amodu via cfe-commits]

ayokunle321 wrote:

> > Tested in our internal workload and the result looks not bad (no crash). This is not a LGTM and I want to take a close look for this. Thanks for your patience in ahead.
> 
> Can you provide maybe some performance numbers, too?
> 
> @ayokunle321, the other thing I was thinking about is that now when we can detect duplicates maybe we can issue better diagnostics advising users how to avoid such duplications.

Yeah, I see from the crash in https://github.com/llvm/llvm-project/issues/127561 there's already a note that tells you the file doing the biggest damage. 

And from the discussions in https://github.com/llvm/llvm-project/issues/139453 there's a suggestion to wrap these troublemakers in a module and import them instead which would get us closer to a more bottom-up build. 

So should the diagnostic should tell them to wrap a heavily duplicated file in a module and import instead? 

https://github.com/llvm/llvm-project/pull/209795