[PATCH] D126800: Write output sections in parallel

[Nico Weber via Phabricator via llvm-commits]

thakis added a comment.

In D126800#3566774 <https://reviews.llvm.org/D126800#3566774>, @michaeleisel wrote:

> @thakis how would #1 look exactly? For example, in `StubsSection::writeTo`, it appears to end up in `inline void writeStub(uint8_t *buf8, const uint32_t stubCode[3], const macho::Symbol &sym)`, which calls `encodePage21`, which calls `checkInt`. Would we make a separate version of `encodePage21` and duplicate the logic, where this new version takes a mutex and a vector along with the usual arguments, and instead of calling normal `checkInt`, it calls our own version that would insert into the vector? Do we duplicate the logic of `reportRangeError`? And would our vector also store the type of message reported, whether it was a warning, error, etc.? What about for fatals, if we happen to find any of those, should we in that case terminate early, and if so, how would that work? With these questions in mind (assuming you don't have some other idea in mind of how to do this), I'm curious what your objections to #2 would be. Also, it seems like #2 would be helpful in any future cases where we want deterministic logging from concurrent tasks, such as if file loading is parallelized.

I haven't looked in detail, but what I would've done is:

- Have some DetermisticParallelDiags class that has warning(), error() methods, and internally a vector to store each
- And a flush() method to make it sort and emit the things it stores
- Have a global object that optionally points to such an object
- Make it point to such an object while we write output sections

This is similar to your 2, but built on top of the existing error messages instead of inside it (which kinda feels more modular to me).

This is just my opinion ofc, if y'all thing this is silly, feel free to ignore!

> Does your build cache the diag outputs as well as the binary outputs?

I think it doesn't matter what our build cache currenty does. Running the same command twice with the same inputs and getting different outputs and getting different output seems Generally Bad, does it not?

(OTOH, if you build with ninja and the build fails, it also doesn't guarantee that errors appear in the same order every time. But for lld, it seems fairly easy to fix, and fixing it if it's easy seems like we should do just do it (?))


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D126800/new/

https://reviews.llvm.org/D126800