<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I'm having a bit of difficulty with the lexer. It's been working dandy for me for the most part, but has some problems when lexing over macro expansions that are empty. <div><br></div><div>Defining this macro:</div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#ifndef MyMacro</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#define MyMacro int g;</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#endif</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">and lexing over this:</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; ">MyMacro <span style="color: #aa0d91">int</span> x;</div></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><br></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">is no problem. But defining the macro thusly:</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#ifndef MyMacro</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#define MyMacro</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#endif</div></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">or:</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#ifndef MyMacro</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#define MyMacro <span style="color: #007400">/*some comment*/</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); ">#endif</div></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">causes a EXC_BAD_ACCESS here:</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; "><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">#0</span></font><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">clang::TokenLexer::isAtEnd at TokenLexer.h:122</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; "><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">#1</span></font><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">clang::TokenLexer::Lex at TokenLexer.cpp:279</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; "><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">#2</span></font><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">clang::Preprocessor::Lex at Preprocessor.h:262</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; "><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">#3</span></font><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">clang::Preprocessor::HandleMacroExpandedIdentifier at PPMacroExpansion.cpp:205</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; "><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">#4</span></font><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">clang::Preprocessor::HandleIdentifier at Preprocessor.cpp:532</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; "><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">#5</span></font><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">clang::Lexer::LexIdentifier at Lexer.cpp:534</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; "><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">#6</span></font><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">clang::Lexer::LexTokenInternal at Lexer.cpp:1314</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; "><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">#7</span></font><span class="Apple-tab-span" style="white-space:pre"><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"> </span></font></span><font class="Apple-style-span" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;">clang::Lexer::Lex at Lexer.h:141</span></font></div><div><font class="Apple-style-span" color="#643820" face="Monaco" size="2"><span class="Apple-style-span" style="font-size: 10px;"><br></span></font></div><div>(Actually, in the comment case, I'm not entirely sure if the comment gets expanded or simply removed by the preprocessor before expansion.)</div><div><br></div><div>This problem seems odd because the source file is getting parsed by clang correctly the first time around (for instance, I can run the clang driver and emit HTML, no problem). But when I make a new lexer (re-using the preprocessor from the original parsing), I get the above errors. Is there some lexer or preprocessor setting that I'm missing?</div><div><br></div><div>Thanks,</div><div><br></div><div>Emerson</div><div><font class="Apple-style-span" color="#643820" face="Monaco" size="2"><span class="Apple-style-span" style="font-size: 10px;"><br></span></font></div></span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 10px/normal Monaco; color: rgb(100, 56, 32); "><font class="Apple-style-span" color="#000000" face="Helvetica" size="3"><span class="Apple-style-span" style="font-size: 12px;"><br></span></font></div></div></body></html>