[PATCH] D60974: Clang IFSO driver action.

Tue Apr 23 13:45:35 PDT 2019

plotfi added a comment.

In D60974#1476084 <https://reviews.llvm.org/D60974#1476084>, @jakehehrlich wrote:

> Why is this better than producing output from the output of the static linker at the end? Also how do we plan on integrating this with llvm-elfabi so that we don't have unnecessarily duplicated efforts? Why does that tool not suffice?
>
> > I'm well versed in the complexities of a linker - I've worked extensively on the GNU linkers as well as with lld. The visibility of the symbols is actually computed by the compiler - the STV_* flags associated with the symtab entry give you that information which is actually tracked through the frontend to the backend. Yes, each linker behaves differently - including lld which completely changes the semantics of Unix/ELF linking guarantees. In fact every single attribute that you mentioned is something which is emitted from the compiler - which we have access to here given that we are in clang itself. I think that this can be made to work properly, though will probably require some iteration to get all the corner cases, and that you may be slightly dismissive of this approach.
> > 
> > Ah, okay, I didn't pick up that you were using module as the actual module from the context since normally the module doesn't control what it exposes itself.
> > 
> > Note that I am not advocating that this change go in as is - I think that this is far from something which would be useful, and until it can produce something meaningful, this shouldn't really be merged (i.e. generate something which obj2yaml or some other tool can consume to generate the barest ELF stub).
>
> It's not a matter of where they're produced, its how you treat them and ensure that they're correctly handled so that the final produced stub has the correct output. I don't think this is entirely trivial when you consider all of those different aspects that I listed.

I'll let @compnerd respond to the above...

>> No, I do not intend to write any sort of linker in the classical sense. At most I'd be handling and merging symbol names from yaml files or possibly going from yaml -> elf .o -> .so. Weighing options.
> 
> I'm calling that a kind of linker here. I'm not saying it would be as complicated as a full linker but it would be a linker. I am claiming it would be more complicated than the dwp linker however. Whatever you're calling it you have to write it and that's an important part of this proposal process but that wasn't mentioned until I asked about it.

You are right about this, because the original proposal was proposing a header -> .so file processor. But because of the issue of faithfully reproducing the ifso based on the build flags, I decided to re-implement things using the clang driver which results the need to do a symbol merge. I could do another proposal RFC if you think that is necessary.

>> I was thinking that since -emit-ifso doesn't invoke the backend, it could be run as a pre-build step. Then during the build, linking is done against the ifso stubs that are already generated instead of the actual .so files.
> 
> That seems a lot better than what I understood. This behavior might be useful to some people. In particular you don't need .tbe files in your source tree so if that isn't something you want to have this would be a less ideal alternative because it was to invoke enough of the compiler to get rich symbol information but not so much that its as slow as compiling. You pay for this by having to reparse and go though some of the compilation process and you have to link/merge many files to produce the stub but it doesn't require funny files in your tree. I think that's a niche that some people might want. The tradeoff between not having to deal with .tbe files and the huge increase of build steps, computation needed (overall build times could very well decrease if your build wasn't sufficiently parallel due to linking orders), and total amount of IO seems like a tradeoff that I personally wouldn't want to make. Is this a trade off you want to make? Almost every metric I can think of would be worse than just adding .tbe files into your source code.

I actually don't see why something like this couldn't be used in both scenarios (ie some users generating the files at build time, and others using -emit-ifso to periodically generate and land changes to checked in files).

Repository:
  rC Clang

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D60974/new/

https://reviews.llvm.org/D60974