[cfe-dev] [RFC] Generate CC1 Command Lines from a CompilerInvocation instance

[Daniel Grumberg via cfe-dev]

I am a PhD student at the Imperial College London currently contracting for Apple, working with Michael Spencer and Duncan Exon Smith on providing build system support for explicit module builds through clang-scan-deps. As part of clang-scan-deps we need to provide build systems with a dependency graph of the modules needed for a given build, as well as a command line that can build each needed module explicitly. Under the hood, clang-scan-deps uses the implicit module build mechanisms to discover modular dependencies. In this setup when we need to generate a command line to build an explicit module we use the command line associated with the TU that discovered the given module and append to it “-remove-preceeding-explicit-module-build-incompatible-options” followed by the options needed to build the module explicitly. The trouble with this is that the command line we generate for building an explicit module is not deterministic, it depends on which TU discovers the module first. Furthermore, this scheme makes it extremely difficult to remove options that are irrelevant to the compilation of the module. What we really want to do is to be able to generate a cc1 command line from first principles given a CompilerInvocation instance.

In order to achieve this goal, we need a strategy that allows us to synthesize command line arguments from the various fields of a CompilerInvocation instance. Unfortunately, CompilerInvocation does not currently support this use case. We propose to automate the process of generating a command line by embedding information inside the table-gen option description files. This extra information will define the mapping between a given compiler option and the fields of CompilerInvocation using a mechanism similar to Obj-C’s keypaths. We can then use this to generate code that automatically translates from a given option to the correct value of the associated CompilerInvocation field and back. A patch describing the proposed pattern is available at https://reviews.llvm.org/D79796 <https://reviews.llvm.org/D79796>. Of course this scheme can not work for complicated mapping logic but we feel that it can automate generating command line arguments and parsing options for the vast majority of cases, which would significantly reduce the amount of custom code inside CompilerInvocation. To make sure that developers keep the mappings up to date, we plan to add an assertion at the end of CompilerInvocation::CreateFromArgs that checks that we can roundtrip back the CompilerInvocation through CompilerInvocation::GenerateCC1CommandLine. This should flag failures in debug builds which would make developers aware of the need to keep the mappings up to date without adding additional runtime overhead to clang.

Thanks for reading and looking forward to everyone’s comments!

Daniel Grumberg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20200512/83785e7d/attachment.html>