<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/139297>139297</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            Preprocessor inconsistency of compiler directives in macros
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            bug,
            flang
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          Meinersbur
      </td>
    </tr>
</table>

<pre>
    Flang fails to parse its own preprocessor output:
```sh
$ flang -E -Xflang -fno-reformat ~/src/llvm-project/flang/module/ieee_arithmetic.f90 -o - | flang - -fsyntax-only
error: Could not parse -
standard input:105:3: error: expected declaration construct
  elemental logical function ieee_is_finite_a2(x); real(2), intent(in) :: x; !dir$ ignore_tkr(d) x; end function ieee_is_finite_a2; elemental logical function ieee_is_finite_a3(x); real(3), intent(in) :: x; !dir$ ignore_tkr(d) x; end function ieee_is_finite_a3; elemental logical function ieee_is_finite_a4(x); real(4), intent(in) :: x; !dir$ ignore_tkr(d) x; end function ieee_is_finite_a4; elemental logical function ieee_is_finite_a8(x); real(8), intent(in) :: x; !dir$ ignore_tkr(d) x; end function ieee_is_finite_a8;
    ^
standard input:2:1: in the context: specification part
  use __fortran_ieee_exceptions
  ^
standard input:1:1: in the context: module
 module ieee_arithmetic
  ^
standard input:106:5: error: expected end of statement
  end interface ieee_is_finite
      ^
[...]
```

The failing like comes from
https://github.com/llvm/llvm-project/blob/a6385a87a2e5537f0790494ebe8bb4c3cc9506b9/flang/module/ieee_arithmetic.f90#L342

which after preprocessing expands to
```f90
elemental logical function ieee_is_finite_a2(x); real(2), intent(in) :: x; !dir$ ignore_tkr(d) x; end function ieee_is_finite_a2;
```

Flang manages to parse this if the the `!dir$` is within a macro, but not if expanded. The error comes from interpreting `!` as a comment line, which removes everything following on the same line from the cooked character stream, including `end function ieee_is_finite_a2;`, which causes the error. My initial diagnosis was that `Prescanner::IsCompilerDirectiveSentinel` does not recognize `!` as a potential start of a compiler directive, but this does not fix the inconcistencies:

```f90
module m
 contains

! Directive inside macro on same line; works
#define MACRO(X)  subroutine func1(X);    real(2) :: X; !dir$ ignore_tkr(d) X; end subroutine func1;
MACRO(foo)

! Same subroutine, but after preprocessor expansion (-e -fno-reformat); syntax error
  ! subroutine func2(foo);  real(2) :: foo; !dir$ ignore_tkr(d) foo; end subroutine func2;

! Parses with line wrap before !dir$
  subroutine func3(foo);     real(2) :: foo;
  !dir$ ignore_tkr(d) foo; end subroutine func3;

! Parses with line wrap after !dir$, but swallows the directive
 subroutine func4(foo); real(2) :: foo; !dir$ ignore_tkr(d) foo;
  end subroutine func4;

end module
```
which results in the following parse tree (`-fdebug-unparse` without func2):
```f90
MODULE m
CONTAINS
 SUBROUTINE func1 (foo)
  REAL(KIND=2_4) foo
  !DIR$ IGNORE_TKR (d) foo
 END SUBROUTINE func1
 SUBROUTINE func3 (foo)
  REAL(KIND=2_4) foo
  !DIR$ IGNORE_TKR (d) foo
 END SUBROUTINE func3
 SUBROUTINE func4 (foo)
 REAL(KIND=2_4) foo
 END SUBROUTINE func4
END MODULE
```
</pre>
<img width="1" height="1" alt="" src="http://email.email.llvm.org/o/eJzUV09v2z4S_TT0ZWBDpiRbOvigxPEi-LVJkbZAbwZFjWxuZNIgqcTZw372xVBS4tjuv110gR_QAnI05HvzZt6QEs6pjUZcsPSKpcuRaP3W2MVHVBqtK1s7Kk31slg1Qm-gFqpx4A3shXUIyjswzxr2FvfWSHTOWDCt37eexQWLCjaLun9uS794AnXYZ3wD42_9Y63N2GJt7E54-DfjK2cl46umedqN99b8E6VnfBWCGV_tTNU2yPhKIeJaWOW3O_RKTuo8grGBMbD59YAC49q9aC8OY6ObFxYVaK2xLC7g2rRNBdr4PpMxiwrnha6ErUDpLoFplLK4iCn-dSEe9ig9VlChbIQVXhkN0mjnbSs9iwoAbHCH2osGGrNRUjRQt1qGwEBauXWttPK4Fpzx7MB4zuIrsCgaxjNOP_k1KO1Re8YzpRnPgfSMCzhQJOPTSllSU220sbj2j5bxrKK4EIC6-hEmRfw6x_icY_ynOca_yTE555j8aY7Jb3LMzjlmf5pjxuKr0JMALL251OOc-pzQlAa_RWpljwd6A26PUtVKdj2-F7Zr79YhrNe1sd4KvQ6IeJC4pygXIr4DNf0-VO_qaHiCE3f_cNtoxuIivexSUsfU4LzwoVCdQXUVJLe1kHiiWa_WgMbSq8lkwtLl8TCj56j4ssUwD5XeQKMeKZ8dOqit2bGo2Hq_d1RIvmJ8tVF-25YTaXb9ZDsfcGVjSsZXYhZnqcjmgmOaxvM6mudRkidYYlaWiYylzNNoVua_OBIZjz_ECe8YP2-V3IKoPdqjkU388bAXuqLJfpwnrY-Kv8E4Oy9Od1zthBYbPDqw_FY5UHVoP_pP8QM6m0WgHDwrv1UaBOyEtIaol60PJ4Wqe52wmgBVP7TbUdm7ptpb9KRptzftKhwICiMZoVEaadeuGBZ35gkd4BPaFwLeQG2axjzTk-l84sQOw7IOpLOOecQK5FZYIamczlsUu05o2bRVT-DnyhHJgYwUrSO1htQm8PEFKFiJBiolNto4UkhQjPAE8Mmik0JrtF1Fb9212e1Vg3apLEqvnvAzaq80NqREZdAFLS1Ks9HqX3gq095QnxCg88J6Mm_QLuwJ1bDpUJdQ0Ndda3UI7JWWRkvlPGqp0PV3kbPO7mcN2TUMI6HCAAs3lSm8JgBKO1Vh1xBUlNeCUHM-G_sYVvG4wpqq9LG4frhnPPtGDQyuLa1pfShfq-W0f0FLAY5dMnji28888W3wxNnWwQkDfG0M4bzm85lYvy0ZFDwdB8Z2Xe6oZRjPxvj-itZz7-5V_cQN05lPTwnxNxaU7YVc6e1Psu1DLuQ7OL9L7xMZvLNvZ5ZnK_ZQEmt8AwhUT_aJ39O8XJWOxpDpf8E2_gW2XS3eyPYlcs-CRkLnzDcLRMUpRvIuk_9F79dz8gxhyIJevh7bx9N3GGyubbwbzvq3odYPYotUlYzNonFdYdluxq0Or2gSkCym9UMT5ScfE517P94vv364Ce69vr_7UtzefSbWn79ePdx__XJ7d9OZAo6tAPBwU3xgPPvr9m7J4iVfJ0POfWWXtw-ky-0_7u4fbtZf_nqAY2GiAm7ulmcYF3Dj_wNufAE3eY_7I9gLO5I_6M-dtseaj6pFXOVxLka4mM6TWTbleZaOtotYyBRrEfOoFCKeTfMonWEqxTyP61kdzUdqwSOeRmmUT-dJnqQTmZV5iulMxtM8TucpSyLcCdVM6EY0MXYzUs61uJjGOc_no0aU2Ljwecp52W4Y54xfM877-w-nr1a7CNepst04lkSNct69beeVb3Dx6XjEhRPCdSfECx0y50dM6N0w892otc3it290IQvH-KpP5GnB_xMAAP__UmPeEg">