[llvm] [LLVM] add LZMA for compression/decompression (PR #83297)

[Artem Belevich via llvm-commits]

Artem-B wrote:

> I was thinking something even more domain specific 

That sounds like a reasonable research project topic. :-)
I suspect there may be something to be gain compression wise for specific ISAs, but I would not expect miracles as existing compression algorithms are pretty good at removing redundancies. zstd do have ability to 'train' on particular set of input files and provide a dictionary optimal for those binaries. It may be interesting to experiment with that.

> maybe some kind of macro scheme to describe the architectural differences,

That's largely what happens in CUDA and AMDGPU. Unfortunately, those minor differences percolate through the rest of the code and we usually end up with similar, but not identical compiler outputs, and it's hard to generalize which parts will be affected, so in practice we do need to compile everything.

> Like do we have population data about kernel sizes? 

Anecdotally, individual kernel size varies from nothing to O(megabytes). Individual TUs (I think that's what object bundler ends up dealing with) will likely be on the smaller side, but the outliers are fairly common. 

> Does wlog=25 cover the 90% case?

My guess is that it should be sufficient for most of the use cases. 


https://github.com/llvm/llvm-project/pull/83297