[llvm] [llvm-rc] Concatenate consecutive string tokens (PR #68685)

[Jacek Caban via llvm-commits]

cjacek wrote:

This way of storing strings and re-parsing it later seems a bit fragile to me. For example, if I change your accelerators-split-token.rc test to use something like: `"^" /* "b" */ "a", 18`, it will crash on assert. Is memory allocation in the tokenizer a no-go?

https://github.com/llvm/llvm-project/pull/68685