[cfe-dev] [RFC] Generate CC1 Command Lines from a CompilerInvocation instance

[Reid Kleckner via cfe-dev]

Broadly speaking, that sounds great. The cc1 interface is unstable, and we
can change it to make the mapping from internal state to command line and
back simpler. The driver code would benefit much more from this kind of
refactoring, but the driver flag handling is so idiosyncratic that this
isn't really feasible.

My input on working with tablegen is: try to generate a little C++ code as
humanly possible. Please try hard to generate readonly data instead of
code. It compiles much more quickly, and usually runs just as quickly.

Our codebase has a bit of a legacy of table generated code gone wrong, and
I want to move away from that where possible. See these (stale) stats on
the individual functions that take the longest to compile:
https://reviews.llvm.org/P8191$95 llvm::decodeToMCInst
They are generated here:
https://github.com/llvm/llvm-project/blob/master/llvm/utils/TableGen/FixedLenDecoderEmitter.cpp#L964
I have tried to optimize the code to compile faster, but so far I've had no
luck.

On Tue, May 12, 2020 at 11:07 AM Daniel Grumberg via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> I am a PhD student at the Imperial College London currently contracting
> for Apple, working with Michael Spencer and Duncan Exon Smith on providing
> build system support for explicit module builds through clang-scan-deps. As
> part of clang-scan-deps we need to provide build systems with a dependency
> graph of the modules needed for a given build, as well as a command line
> that can build each needed module explicitly. Under the hood,
> clang-scan-deps uses the implicit module build mechanisms to discover
> modular dependencies. In this setup when we need to generate a command line
> to build an explicit module we use the command line associated with the TU
> that discovered the given module and append to it
> “-remove-preceeding-explicit-module-build-incompatible-options” followed by
> the options needed to build the module explicitly. The trouble with this is
> that the command line we generate for building an explicit module is
> not deterministic, it depends on which TU discovers the module first.
> Furthermore, this scheme makes it extremely difficult to remove options
> that are irrelevant to the compilation of the module. What we really want
> to do is to be able to generate a cc1 command line from first principles
> given a CompilerInvocation instance.
>
> In order to achieve this goal, we need a strategy that allows us to
> synthesize command line arguments from the various fields of a
> CompilerInvocation instance. Unfortunately, CompilerInvocation does not
> currently support this use case. We propose to automate the process of
> generating a command line by embedding information inside the
> table-gen option description files. This extra information will define the
> mapping between a given compiler option and the fields of
> CompilerInvocation using a mechanism similar to Obj-C’s keypaths. We can
> then use this to generate code that automatically translates from a given
> option to the correct value of the associated CompilerInvocation field and
> back. A patch describing the proposed pattern is available at
> https://reviews.llvm.org/D79796. Of course this scheme can not work for
> complicated mapping logic but we feel that it can automate generating
> command line arguments and parsing options for the vast majority of cases,
> which would significantly reduce the amount of custom code inside
> CompilerInvocation. To make sure that developers keep the mappings up to
> date, we plan to add an assertion at the end of
> CompilerInvocation::CreateFromArgs that checks that we can roundtrip back
> the CompilerInvocation through CompilerInvocation::GenerateCC1CommandLine.
> This should flag failures in debug builds which would make developers aware
> of the need to keep the mappings up to date without adding additional
> runtime overhead to clang.
>
> Thanks for reading and looking forward to everyone’s comments!
>
> Daniel Grumberg
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20200513/d6fbb58e/attachment.html>