<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body><p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;"><span style="font-size:0.75em;">Just a short feedback on that. I played a bit with you code and got the needed information seems to be </span></p>
<br /><p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">void MacroExpands(const Token &MacroName, const MacroDefinition &MD, SourceRange Range, const MacroArgs *Args) override {</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;"> for (unsigned i=0;i<Args->getNumMacroArguments ();++i){</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;"> Args->getUnexpArgument(i)->getLocation().printToString(SM)<<":"<<Args->getUnexpArgument(i)->getEndLoc().printToString(SM)<<"\n";</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;"> }</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">}</p>
<br /><p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">I went ahead and basically copied and pasted your code in order to add a new collection for unexpanded arguments.</p>
<br /><p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">In contrast to your code the tokenWatcher does not make sense in this regard since it will probably lex the preprocessed hence expanded source code.</p>
<br /><p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">Thanks for your work</p>
<p> </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">On Donnerstag, 26. August 2021 17:00:09 CEST chiasa.men via llvm-dev wrote:</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> Thanks you were right, that fixed the minor printing issue.</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> But the actual problem of string literals in macros still persists. Maybe</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> someone else has an idea on that. The information isn't in reach of your</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> MacroExpansionContext/MacroExpansionRangeRecorder, is it?</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> On Donnerstag, 26. August 2021 12:32:22 CEST benicsbalazs@gmail.com wrote:4</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Oh, now I see why you were puzzled!</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > `printf()` expects a null-terminated string (“%s”), but you pass an</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > llvm::StringRef instead, which is not null terminated!</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Thus, you will end up with undefined behavior, which ends up printing till</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > the end of the file.</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > I hope this is it!</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Balazs</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > From: benicsbalazs@gmail.com <benicsbalazs@gmail.com></p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Sent: 2021. augusztus 26., csütörtök 12:25</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > To: 'chiasa.men' <chiasa.men@web.de></p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Cc: llvm-dev@lists.llvm.org</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Subject: RE: [cfe-dev] Location (in Source Code) of String Literal in</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > MacroExpansion</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Hi Chiasa,</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > For the example you attached the MacroExpansionContext seems to work as I</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > intended.</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > It records what range gets replaced by what tokens at the end of the</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > preprocessing, thus we get</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > `CONCATMACRO("TEST")` -> ` L"TEST"`</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > About `Lexer::getSourceText()` I’m not sure. I haven’t used that. The</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > `at()` function was designed as a helper defining the unit tests. I might</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > misused something when I created this code & tests, let me know!</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Balazs</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > From: cfe-dev <cfe-dev-bounces@lists.llvm.org</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > <mailto:cfe-dev-bounces@lists.llvm.org> > On Behalf Of chiasa.men via</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > cfe-dev Sent: 2021. augusztus 26., csütörtök 10:03</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > To: cfe-dev <cfe-dev@lists.llvm.org <mailto:cfe-dev@lists.llvm.org> ></p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Cc: llvm-dev@lists.llvm.org <mailto:llvm-dev@lists.llvm.org></p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Subject: [cfe-dev] Location (in Source Code) of String Literal in</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > MacroExpansion</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > https://gist.github.com/T35R6braPwgDJKq/6439fda090e4ee8440d726f8dafa4dbb#f</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > il e-mectest-cpp</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Above I modified the MacroExpansionContext unittest.</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > The relevant code:</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > //--------------</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > TEST_F(MacroExpansionContextTest, Custom) {</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > // From the GCC website.</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > const auto Ctx = getMacroExpansionContextFor(R"code(</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > #define CONCATMACRO(parm) L##parm</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > int f(wchar_t *b){}</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > int main()</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > {</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > int a= f(CONCATMACRO("TEST"));</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > int b= f(L"TEST");</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > return 0;</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > }</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > )code");</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Ctx->dumpExpansionRanges();</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > //> :6:14, :6:33</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Ctx->dumpExpandedTexts();</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > //> :6:14 -> 'L"TEST"'</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > printf("Exp: %s\n",Ctx->getExpandedText(at(6, 14)).getValue());</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > //Exp: L"TEST"</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > printf("Org: %s\n",Ctx->getOriginalText(at(6, 14)).getValue());</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > //Org: CONCATMACRO("TEST"));</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > // int b= f(L"TEST");</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > // return 0;</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > //}</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > StringRef sourceText =</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > clang::Lexer::getSourceText(CharSourceRange(SourceRange(at(6, 14),at(6,</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > 33)),false),SourceMgr, LangOpts);</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > printf("sourceText: %s\n",sourceText);</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > // sourceText: CONCATMACRO("TEST"));</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > // int b= f(L"TEST");</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > // return 0;</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > // }</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > }</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > //--------------</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > I am interested in getting the range for "TEST" in the SourceText. Since</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > the CONCATMACRO is preprocessed to L"TEST" I thought the</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > MacroExpansionContext would be of help.</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > It tells about the ExpansionRanges: 6:14, :6:33 which is</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > CONCATMACRO("TEST")</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > What I would need is: 6:26, 6:32</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Obviously I could just calculate a bit like:</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > range.begin (14) + macronameLen (11) +1 (opening parenthesis) = 26</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > to</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > range.end (33) - 1 (closing parenthesis) = 32</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > But would that be reliable and is there no other way?</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > </p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > Furthermore: Why do getSourceText/getOriginalText print the whole rest of</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;">> > the source code and not just from (6,14) to (6,33)?</p>
<p style="margin-top:0;margin-bottom:0;margin-left:0;margin-right:0;"><br /></p>
</body>
</html>