[clang] Llvm modules on demand bmi (PR #71773)

[Chuanqi Xu via cfe-commits]

ChuanqiXu9 wrote:

> > > > > > > Whether we re-use wrapper code or make some new code is an implementation detail.
> > > > > > > It does not actually prevent you from taking the two-phase approach ( currently , --precompile will be unchanged in action).
> > > > > > 
> > > > > > 
> > > > > > It doesn't prevent the two-phase compilation model indeed. But it introduces a new way about how we produce BMIs.
> > > > > 
> > > > > 
> > > > > It allows us to produce a new kind of BMI - that carries a minimised content, applicable to the interface, otherwise it is no different to the case where two command lines are needed to produce an object and BMI..
> > > > 
> > > > 
> > > > This is what thin BMI or interface BMI does.
> > > 
> > > 
> > > Agreed - but the actual process of producing the interface BMI means taking the "full" AST (which is necessary to generate the object) and reducing it by applying suitable filters [like in my diagram].
> > > This is done in an instance of the front end - it has either to be provided with the full AST or have a multiplexer that puts the full AST to code-gen and then applies filtering to the other path.
> > 
> > 
> > Yeah, the full AST is necessary to produce the object file. And the ASTWriters can choose to not write parts of the AST to disk.
> 
> I am still strongly against making the filters part of the serialization process, we have learned that this is often a long-term mistake (e.g. doing decl merging in the AST reader).
> 
> The filtering should be thought of as as AST -> AST transform and the AST serializer should just write what is being given to it.
> 
> but, in either case this implies that the original AST is split into two paths - one going to the code-gen and one going to the BMI output; that is what this patch series is doing, I believe.

We've already known that we don't have a good API to discard part of the AST from the experience in implementing discarding decls in GMF. Then we'd have to control it in the AST Writers.

> 
> > > > > The difficulty that I have pointed out is that if we preserve the existing scheme but want an Interface BMI - we then have to produce a third compile line in the driver that takes the Implementation BMI and produces the Interface BMI from it. We cannot avoid producing the intermediate BMI here because the jobs are created by the driver and executed by the compiler and we need to Implementation BMI to produce the object.
> > > > 
> > > > 
> > > > Oh, this may be the root of the divergence here. In my mind, we can make it without producing new compile jobs. I've already looked at the code. We can avoid producing the intermediate BMI by skipping some phases in the drivers.
> > > 
> > > 
> > > I still do not understand the point here: the driver is not doing the work; the driver is preparing cc1 command lines - skipping phases does not achieve anything unless the front end is capable of producing both the BMI and object at the same time (any other solution means materialising the full BMI as an entity).
> > 
> > 
> > My point is that we need to introduce a new frontend action. And the driver's job is to create the new frontend action.
> 
> It is not one action since a BMI could be combined with any other code-gen action. Perhaps all that is needed is for the driver to add a `--bmi` flag to cc1 jobs that code-gen ?

No, it can be an action and we can control the usage of the BMI.



https://github.com/llvm/llvm-project/pull/71773