[llvm] [llvm-rc] Concatenate consecutive string tokens (PR #68685)

[Martin Storsjö via llvm-commits]

mstorsjo wrote:

> The usual behavior (like in C) is that such concatenation promotes the whole string to be a wide string and testing with windres confirms that it follows that pattern. I used following test:
> 
> ```
> 1 test
> {
>    "aaaa",
>    L"bbbb" "cccc",
> }
> ```
> 
> windres will interpret `"aaaa"` as a narrow string, but `L"bbbb" "cccc"` as a wide string, the same as `L"bbbbcccc"`. Implementing that behavior seems right to me and it means that the concatenation in tokenizer would not need to preserve 'L' part of each component separately and should be able to just concatenate the payload.
> 
> BTW, `"bbbb" L"cccc"` is considered a syntax error by windres.

Oh, right, that explains things. Thanks!

`rc.exe`, which generally doesn't concatenate string tokens but only allows consecutive string literals in some contexts, interprets it differently; with your example, `L"bbbb" "cccc"` `"cccc"` ends up as a narrow string. So to achieve the GNU windres behaviour, we don't need to maintain the `rc.exe` behaviour - and then this becomes much much easier to implement just as you're suggesting. Thanks!

So I guess that leaves us with two choices:

- We add a windres/rc distinction into the general parsing process in llvm-rc, and emulate both behaviours. (Currently we have the `rc.exe` style behaviour, so we'd just need to make the new token merging optional.) Slightly sad that we'd have to do it as we've managed without doing that so far, but it's probably not a big deal.
- We conclude that `L"bbbb" "cccc"` actually meaning a narrow `"cccc"` in `rc.exe`, or conversely being able to do `"bbbb" L"cccc"` as is possible with `rc.exe` (and llvm-rc right now), is a theoretical corner case, and nobody probably is doing that in practice.

Do we have an opinion on this from users of llvm-rc in MSVC-like environments - @zmodem @nico @aganea, or others?

Cautiously, I would prefer the former - faithfully emulating `rc.exe`'s quirks when invoked as llvm-rc, even if the corner cases are kinda theoretical.

https://github.com/llvm/llvm-project/pull/68685