[llvm] [Opt] Enable statically-linked plugin support (PR #79227)

[William Moses via llvm-commits]

wsmoses wrote:

So the way the clang one worked, was as follows:

Bazel's build for clang already separated all of the clang driver libraries into a separate `clang-driver` library (https://github.com/llvm/llvm-project/blob/ff05c3087b4b7e0121125d7f6860b3d9383ca91d/utils/bazel/llvm-project-overlay/clang/BUILD.bazel#L2307) which then to build clang in Bazel one just makes an executable with it (https://github.com/llvm/llvm-project/blob/ff05c3087b4b7e0121125d7f6860b3d9383ca91d/utils/bazel/llvm-project-overlay/clang/BUILD.bazel#L2298). 

For our custom clang, we simply linked the clang-driver against whatever plugin we wanted.

Doing the same for opt has two issues:
1) There is no opt-driver like library (instead the bazel executable contains all of the functionality).
2) Opt plugins are only loaded by passing in a path to a .so to load, and do not have an alternate registration mechanism (e.g. registry, list of args, other global, etc).

Originally the first Bazel PR (https://github.com/llvm/llvm-project/pull/79205) was a 2-line bazel patch which added a similar opt-driver library target in Bazel, like there exists already for clang. Code review requested that if this was going to happen, one should also make a library for the opt driver in CMake as well.

Unfortunately making such a library in CMake is a non-trivial task, if one wants to permit all build targets (e.g. shared libs, etc), which obviously we need to support. Specifically, it required creating a separate opt.cpp file which just contains main, as distinct from the actual opt functionality. This was required to make shared library linkage work (among other minor things, like not linking the new lib into libLLVM, etc).

Since the goal was to support additional plugins, we went ahead and just added the vector of callbacks to the optMain function in the new library. 

We could also support a registry, and remove the vector of callbacks to optMain. However, the need for the separate opt.cpp with just main for each executable variant [e.g. actual opt, this test plugin opt], is required for CMake + build variants.

In light of that I'd say the two most reasonable design decisions are to either:
1) pass the plugins as a vector to optMain, since we already have to have an optMain for the cmake library
2) add a registry, but remove support for CMake and only allow it for Bazel.

1) is nice because all LLVM users can do something like this. Also incidentally this is somewhat similar to how this works for mlir with mlirOptMain (https://github.com/llvm/llvm-project/blob/f826f55b2ab68c2515fae751dc2d6ef77f37b172/mlir/lib/Tools/mlir-opt/MlirOptMain.cpp#L504). However, this is different from clang, and each downstream user needs to have their own clang
2) is code-wise simpler, but only works for Bazel.

https://github.com/llvm/llvm-project/pull/79227