[flang-dev] [RFC] Frontend driver: action vs non-action options

Thu Oct 14 14:21:04 PDT 2021

Since we can only have one action option at a time, any option that might reasonably appear in combination with another option should not be an action option.  As Michael noted, we have many dump options, and I personally use them routinely in various combinations.  This, to me, argues that none of them should be action options.

Things in the compiler proceed in a linear series of operations -- preprocessing, parsing, semantic analysis, code generation.  It make sense to me to have the options that control this sequence to be action options.  I would think that everything else would be flag options.

I agree with Michael that conflicting options should produce error messages.  I think that this should apply to options generally, not just action options.

Pete

-----Original Message-----
From: flang-dev <flang-dev-bounces at lists.llvm.org> On Behalf Of Andrzej Warzynski via flang-dev
Sent: Thursday, October 14, 2021 5:27 AM
To: Michael Kruse <llvm at meinersbur.de>
Cc: via flang-dev <flang-dev at lists.llvm.org>
Subject: Re: [flang-dev] [RFC] Frontend driver: action vs non-action options

External email: Use caution opening links or attachments

Michael, that's a very re-assuring reply, thank you! Also, these are very helpful and much appreciated suggestions.

On 13/10/2021 19:32, Michael Kruse wrote:
> IMHO the compiler should error-out if multiple (different) actions are 
> specified.

Here's my naive attempt to implement this behaviour:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freviews.llvm.org%2FD111781&data=04%7C01%7Cpsteinfeld%40nvidia.com%7C24f996a3c85740e4e4d708d98f0e0968%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637698112733879280%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2F7QO4rL1vlH%2FODd2bJa%2BsWMnmdye%2BRR7jHZRUsdZ%2FIg%3D&reserved=0. This extends the ArgList API rather than the TableGen logic. It has its limitations, so I'm not sure whether that's the right approach.

I really like your idea of `-fdump=unparse,symbols -o dump.txt`. It would nicely extend the current approach, so might be the easiest path towards more flexible "dumping" options.

-Andrzej

> If one wants to have multiple outputs (such as the AST dump as well as 
> the assembly output) one needs to invoke the compiler multiple times. 
> An alternative would be to have options that specify additional output 
> file, as -MF does (i.e. -M/-MF is not an action but an option).
>
> In some contexts it may make sense to allow contradicting options on 
> the command line as a result of a build system combining multiple 
> sources for flags. For instance, CMAKE_BUILD_TYPE=Release will add -O3 
> by default, but I can override this with e.g. CMAKE_CXX_FLAGS=-O2. I 
> don't think this makes sense with actions as the output what the build 
> system expects would be completely different.
>
> Thanks to Andrej's list, there are quite some dump options. Maybe it 
> makes sense to have a single dump action (llvm.org/PR52095) and an 
> additional option for selecting what is to be dumped to avoid multiple 
> invocations. Eg. `-fdump=unparse,symbols -o dump.txt`. Alternatively, 
> like `-M`, have options that specify where the dump output goes. These 
> could be added e.g. CMAKE_CXX_FLAGS to have dumps written next to the 
> main output while using the main build system.
>
> As Andrej mentioned, the clang/flang option parsing system behaviour 
> to ignore all but the last for exclusive options. Maybe we should 
> extend it such that in the .td file we can specify that it is an 
> error/warning if mutually exclusive options are specified?
>
> Michael
>
>
>
> Am Mi., 13. Okt. 2021 um 09:39 Uhr schrieb Andrzej Warzynski via 
> flang-dev <flang-dev at lists.llvm.org>:
>>
>> Hi All,
>>
>> [Flang = LLVM Flang]
>>
>> I wanted to dive a bit deeper into one aspect of the design of our 
>> *frontend driver*, `flang-new -fc1`. It is intended as the main tool 
>> for Flang frontend developers and we should make sure that it meets 
>> our requirements. If not, perhaps we can identify how to make it work 
>> better? More specifically, does the current split into action and 
>> non-action options makes sense?
>>
>> Please note - this discussion does not involve the compiler driver, 
>> `flang-new`.
>>
>> # *Action vs non-action options*
>> In the frontend driver, you can split options into two groups:
>>       * action options (e.g. `-emit-llvm`)
>>       * non-action options (feature/configuration flags, e.g. 
>> `-ffixed-form`) Action options specify "what" the compiler will do, 
>> e.g. "parse the input file and lower it to LLVM IR" for `-emit-llvm`. 
>> Non-action options allow users to configure the frontend to behave in a certain way, e.g.
>> treat all input files as fixed-form, regardless of the file extension.
>> Another example for the latter is `-Werror` - make all warnings into 
>> errors. This tells the compiler "how" the deal with certain 
>> scenarios, but does not specify "what" to do.
>>
>> AFAIK, this design is consistent with `clang -cc1`.
>>
>> # *Default action*
>> The frontend driver will run `-fsyntax-only` by default, i.e. the 
>> following two invocations are identical:
>>
>> ```lang=bash
>> $ flang-new -fc1 input.f95
>> $ flang-new -fc1 -fsyntax-only input.f95 ```
>>
>> If users want a different action to be run, they need to specify it 
>> explicitly, e.g.
>>
>> ```lang=bash
>> $ flang-new -fc1 -fdebug-unparse input.f95 ```
>>
>> This behaviour is consistent with `clang -cc1`.
>>
>> # *Selected Implementation Details*
>> For every action flag, there's a dedicated specialisation of the 
>> `FrontendActions` class in FrontendActions.h [1]. Right now, the only 
>> way to identify which flag is an action flag, is to look up the 
>> implementation. For example, options decorated with `Action_Group` in 
>> Options.td [2] are action flags. Alternatively, you can check the 
>> switch statements in CompilerInvocation.cpp [3] that selects the 
>> frontend action based on the input flags.
>>
>> # *One action at a time*
>> The frontend driver will only run the rightmost frontend action 
>> specified on the command line. This is illustrated below:
>>
>> ```lang=bash
>> # Runs -fdebug-unparse, -fdebug-pre-fir-tree is ignored $ flang-new 
>> -fc1 -fdebug-pre-fir-tree -fdebug-unparse input.f95 # Runs 
>> -fdebug-pre-fir-tree, -fdebug-unparse is ignored $ flang-new -fc1 
>> -fdebug-unparse  -fdebug-pre-fir-tree input.f95 ```
>>
>> This approach means that the design is relatively simple. Allowing 
>> multiple actions per invocation would require the driver to schedule 
>> and to keep track of the requested actions. That would complicate the 
>> implementation.
>>
>> `flang-new -fc1` does not warn about action flags being ignored. This 
>> is counter-intuitive, but I wasn't able to identify an easy fix for this.
>> We've recently discussed this in Bugzilla [4].
>>
>> AFAIK, this behaviour is consistent with `clang -cc1`.
>>
>> # *Does this work for you*?
>>
>> With the "one action at a time" rule, it is crucial to make sure that 
>> the split into action and non-action options is optimal. So far we 
>> have used a few heuristics:
>> * if a flag requires something unique to happen, it's an action (e.g.
>> `-fdebug-unparse` requires a call to `Fortran::parser::Unparse` 
>> that's otherwise not required)
>> * if something is an action in `clang -cc1`, it should also be an 
>> action in `flang-new -fc1` (e.g. `-emit-llvm`)
>> * if a flag configures some state, it's not an action (e.g. 
>> `-ffixed-form`)
>>
>> To make this a bit easier to visualise, I've compiled the list of 
>> options in `flang-new -fc1` in a spreadsheet [5]. I've also added 
>> actions options from `clang -cc1` for comparison. Note that out of 
>> ~820 options in `clang -cc1` (extracted from `clang -cc1 -help`), 
>> only 36 are action options.
>>
>> Note: I've tried to make my spreadsheet [5] as accurate as possible, 
>> but wasn't able to generate it automatically and might have missed something.
>>
>> So, does the current split make sense to you? Should any of the 
>> options be re-implemented? I may not have the time to implement the 
>> suggested changes myself, but will be available to help you and to 
>> review your patches. And I will also be adding more documentation 
>> upstream. In fact, first patch is already available [6].
>>
>> Thank you for taking a look!
>> -Andrzej
>>
>>
>> [1]
>> https://github.com/llvm/llvm-project/blob/main/flang/include/flang/Fr
>> ontend/FrontendActions.h
>> [2]
>> https://github.com/llvm/llvm-project/blob/main/clang/include/clang/Dr
>> iver/Options.td
>> [3]
>> https://github.com/llvm/llvm-project/blob/main/flang/lib/Frontend/Com
>> pilerInvocation.cpp#L111-L178 [4] 
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbug
>> s.llvm.org%2Fshow_bug.cgi%3Fid%3D52095&data=04%7C01%7Cpsteinfeld%
>> 40nvidia.com%7C24f996a3c85740e4e4d708d98f0e0968%7C43083d15727340c1b7d
>> b39efd9ccc17a%7C0%7C0%7C637698112733879280%7CUnknown%7CTWFpbGZsb3d8ey
>> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C10
>> 00&sdata=4W0ThI5xZPrBtOUhThNX6att6TyK7LxYFdwfGD46RYU%3D&reser
>> ved=0
>> [5]
>> https://docs.google.com/spreadsheets/d/1Cbzv58m_bO439sCRa4-onhhi-3JdF
>> nLAhhTkeCMDdD4/edit?usp=sharing [6] 
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Frev
>> iews.llvm.org%2FD111573&data=04%7C01%7Cpsteinfeld%40nvidia.com%7C
>> 24f996a3c85740e4e4d708d98f0e0968%7C43083d15727340c1b7db39efd9ccc17a%7
>> C0%7C0%7C637698112733879280%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
>> DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=59
>> n06GsKbuouASqZH26vZKttj4t%2BnedxQ7beuBhZc64%3D&reserved=0
>> _______________________________________________
>> flang-dev mailing list
>> flang-dev at lists.llvm.org
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flis
>> ts.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fflang-dev&data=04%7C
>> 01%7Cpsteinfeld%40nvidia.com%7C24f996a3c85740e4e4d708d98f0e0968%7C430
>> 83d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637698112733879280%7CUnknown%
>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
>> XVCI6Mn0%3D%7C1000&sdata=GlCmQS2FWplmHBIgfH3L1PyG94HQdv2vhkK%2FL0
>> iwZxw%3D&reserved=0
_______________________________________________
flang-dev mailing list
flang-dev at lists.llvm.org
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fflang-dev&data=04%7C01%7Cpsteinfeld%40nvidia.com%7C24f996a3c85740e4e4d708d98f0e0968%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637698112733879280%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GlCmQS2FWplmHBIgfH3L1PyG94HQdv2vhkK%2FL0iwZxw%3D&reserved=0