<div dir="ltr">Hello Reid,<div><br></div><div>I'm not exactly sure I understand completely your proposal.</div><div>Are you proposing to add overridable virtual methods to MCAsmParserExtensions to be used by AsmLexer to specify which characters are part of an identifier or not?</div><div><br></div><div>If that is the case I'm not sure how MCAsmLexer or AsmLexer can make use of those , because while MCAsmParserExtensions sees MCAsmLexer the latter doesn't know anything about the former.</div><div><br></div><div>Or ... maybe I completely misinterpreted what you were saying? :-D </div><div><br></div><div>Marcello</div></div><div class="gmail_extra"><br><div class="gmail_quote">2014-11-12 10:45 GMT-08:00 Reid Kleckner <span dir="ltr"><<a href="mailto:rnk@google.com" target="_blank">rnk@google.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I think allowing MCAsmParserExtensions to control this behavior by overriding methods would be cleaner than adding more setters. I'm imagining that each target is allowed to supply its own table or virtual method to implement 'IsIdentifierChar' in AsmLexer.cpp. This would handle AllowAtInIdentifier and your use case.</div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">On Wed, Nov 12, 2014 at 9:23 AM, Marcello Maggioni <span dir="ltr"><<a href="mailto:hayarms@gmail.com" target="_blank">hayarms@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr">Hello,<div><br></div><div>I would like to gather some ideas and opinions on how to make the default AsmLexer more flexible when dealing with Identifiers.</div><div><br></div><div>When the lexer emits something as an "Identifier" (read. String of characters) it means that it needs to be parsed all at once in a single go, even if it contains elements that might be wanted to be parsed as separate entities.</div><div>In that case it is needed to implement some custom parsing logic that lexes and parses in place the identifier string to emit the Operands in the operand vector, which might not be ideal.</div><div><br></div><div>At the moment the default AsmLexer lexes tokens like this:</div><div><br></div><div>There are a bunch of symbols that are parsed directly into tokens(like #, % ... etc), then there are integer/float literals and a fairly big category that catches the default case that doesn't match any of the previous, that are handled by the LexIdentifier() function.</div><div><br></div><div>Actually in the current default AsmLexer this function doesn't always emit an Identifier token, but might return Float literals or Dot tokens in some special cases, so it works more like a "handle what I couldn't directly recognize" kind of function.</div><div><br></div><div>In multiple occasions I found like I wanted to be able to change what actually this function considers an Identifier or separate tokens.</div><div><br></div><div>A use case would be this.</div><div><br></div><div>Let's say that my target's assembly syntax has this fancy characteristic where different operands are separated by '$' (dollar) like in:</div><div><br></div><div>add r0$5$r3</div><div><br></div><div>The default AsmLexer would lex the entire r0$5$r3 as a single "Identifier" and it is not possible to Lex every operand separately , but some custom lexing logic must be applied over the returned "Identifier" Token to split and recognize each of the operands.</div><div><br></div><div>This is a stupid example, but there are other cases where something similar happens and can be a hassle to deal with, because what an Identifier is entirely dependent from some arbitrary logic in the Lexer.</div><div><br></div><div>To override this logic the entire default Lexer and Parser needs to be overridden (probably copying most of the existing logic for the rest of the parsing anyway).</div><div><br></div><div>I would like to find a more easy way to specify what to return as an identifier or separate logic allowing for more flexibility.</div><div><br></div><div>I developed a tentative patch that adds this flexibility to the current MCAsmLexer infrastructure.</div><div>I would like to gather opinions on this approach or ideas on other possible approaches to achieve something similar and find out if somebody else finds this kind of concept useful or not.</div><div><br></div><div>Thanks,</div><div>Marcello</div><div><br></div><div><br></div></div>
<br></div></div>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>