[PATCH] D31100: [LTO] Allow client to skip code gen

[Teresa Johnson via Phabricator via llvm-commits]

tejohnson added a comment.

In https://reviews.llvm.org/D31100#704352, @mehdi_amini wrote:

> In https://reviews.llvm.org/D31100#704314, @tejohnson wrote:
>
> > In https://reviews.llvm.org/D31100#704279, @mehdi_amini wrote:
> >
> > > In https://reviews.llvm.org/D31100#704273, @tejohnson wrote:
> > >
> > > > In https://reviews.llvm.org/D31100#704263, @mehdi_amini wrote:
> > > >
> > > > > > We noticed that when invoking the thinBackend via clang (for the distributed build case) that flags like -ffunction-sections and -emit-llvm were not having the intended effect. This could have been fixed by setting up the TargetOptions and the CodeGenFileType in the LTO Config, but since clang already has handling for all of this, it is straightforward to just let it do the handling.
> > > > >
> > > > > Well, I'm not convinced it is the right thing to do. It is the most straightforward thing to do for the distributed case, but it creates a discrepancy with the non-distributed case, and it won't allow to honor -ffunction-sections this way.
> > > >
> > > >
> > > > Can you clarify the concern? We do pass down -ffunction-sections correctly in the non-distributed case, e.g. via the gold plugin. The gold plugin which is only handling LTO doesn't have any built in code gen handling (anymore with the new LTO API).
> > >
> > >
> > > OK, then how do we pass down -ffunction-sections to the LTO API in the non-distributed case? I expect the same flow to happen here.
> >
> >
> > See AddGoldPlugin in clang for where we translate it into the equivalent llvm internal option passed to the plugin. It eventually gets used to init the target options in the Config in the plugin.
>
>
> Yes saw that, this is what I was expecting "somehow".
>
> We're setting this up as a side-channel using `cl::ParseCommandLineOptions(NumOpts, &options::extra[0]);` in gold-plugin.cpp (oh boy I don't like the backend in general, and this is no exception).
>
> And then: `  Conf.Options = InitTargetOptionsFromCodeGenFlags();` (`Conf` is `lto::Config` here, and `Conf.Options` is `TargetOptions`)
>
> So I rather get clang to initialize the TargetOptions in the LTO config. I think it should be easy to extract `EmitAssemblyHelper::CreateTargetOptions` out of `EmitAssemblyHelper::CreateTargetMachine` and reuse it for this purpose.


That is doable, but I don't see the advantage of duplicating the logic in EmitAssemblyHelper (e.g. CreateTargetMachine), just so we can invoke the codegen through LTO, since we have to be able to do this outside LTO in EmitAssemblyHelper in the non-LTO case in clang. We have to set up TargetOptions in either case, see below.

> 
> 
>>> 
>>> 
>>>>> I also believe that it won't help to guarantee that wether you're using a distributed build or not you get the same binary.
>>>> 
>>>> How so?
>>> 
>>> The Target setup can take many options in the way it is created. Having the exact same flow and API used in both cases helps figuring out where any discrepancy comes from (it is also less moving pieces).
>> 
>> We're more limited by what gets passed to the plugin and how, e.g. in the function sections case.
> 
> Not sure what you mean here, I suspect this is why I'm against using a different flow: we're not supposed to be passing flags differently one case from the other.

There's not much choice here I think. In the case of doing a ThinLTO backend via clang, this information is passed in memory in the CodeGenOptions data structure. That is used to fill in the TargetOptions data structure in EmitAssemblyHelper::CreateTargetMachine.

In the in-process gold case, it needs to be passed from clang into the gold plugin somehow, which can't be done directly via the CodeGenOptions data structure, so we pass internal options. Then the gold plugin is responsible for getting those to init the TargetOptions (as you note above it does that via InitTargetOptionsFromCodeGenFlags).

So regardless of whether the clang invocation of the ThinLTO backends uses LTO to do the code gen, and sets up the TargetOptions struct within lto::Config, or just uses the existing EmitAssemblyHelper which sets up the TargetOptions there, the setup of TargetOptions is up to the LTO client.


https://reviews.llvm.org/D31100