<div dir="ltr"><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><span class="im"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div>
2. There may be the case that x is not used directly but together with
operations(or worse a function call). The resulting type in this case
would require that I deduct the type of the operator[] parameter for a
statement under the condition of a changed type of x.<br></div></div></div></div></div></blockquote><div><br></div></span><div>Yep.
I'd first look how many of those there are, often, even in > 100MLOC
code bases, there are only a handful of corner cases.</div></blockquote><div><br></div><div>You are right, after looking through my generated code it doesn't seem to be a problem.<br><br><br><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><span class="im"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>My
hope is still that there is a "keep the node in the ast" solution for
clang that would allow me to rewrite such a statement based on the
parameter type without caring about the exact statement within the
brackets.<br></div></div></div></blockquote><div><br></div></span><div>But
then the question is how can you be sure that it works afterwards? For
that it seems like you'd need to know the exact information you would
get when analyzing the unchanged code first (which is why I suggested
that).</div></blockquote><div><br></div><div>That is why I thought that i may force clang to keep those nodes by declaring empty dummy builtin operator overloads for any pointer type, which I could later replace with the help of the ast. That way I thought I could be sure that the source code transformations are valid.Basically the idea was to use internal clang methods to define something along the lines of:<br><div style="margin-left:40px">template <typename T><br></div><div style="margin-left:40px">T & operator[](T * array, const X & index){...}<br></div>which unfortunately is not a valid c++ statement. Then again I lack knowledge about the inner workings of clang to do that and that idea may be way more complicated/impractical than I thought.<br><br><br></div><div>Again thanks for your help. Your solution should be more practical than trying to handle every theoretical case in a general manner as i first intended.<br><br><br></div><div>Regards,<br></div><div>Marc Greim<br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 9, 2014 at 3:01 PM, Manuel Klimek <span dir="ltr"><<a href="mailto:klimek@google.com" target="_blank">klimek@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br><br><div class="gmail_quote"><span class="">On Thu Oct 09 2014 at 2:57:14 PM Marc Greim <<a href="mailto:marc.greim@mytum.de" target="_blank">marc.greim@mytum.de</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div>First of all thanks for your patience and help with my problem.<br><br></div>Your are right i cannot change the code generator (and it may possibly even be exchanged with another generator but that is future problem).<br><br></div>Your solution seems right to me in the simple case of the example. <br>But I think it fails for me due to the complexity of the generated code for 2 reasons:<br></div> 1. The generated code uses macro defines to declare variables. I only change some defines but i don't know exactly which variables are affected. (still this may be solvable by changing those defines, record the variables, change them back and record the use, or record macro expansion operations for those cases)<br></div></div></div></div></div></div></blockquote><div><br></div></span><div>If you know the locations where you change the defines, you can figure out which variables are affected. Clang has all the information about macro expansion stored in the AST / the source-manager.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div></div> 2. There may be the case that x is not used directly but together with operations(or worse a function call). The resulting type in this case would require that I deduct the type of the operator[] parameter for a statement under the condition of a changed type of x.<br></div></div></div></div></div></blockquote><div><br></div></span><div>Yep. I'd first look how many of those there are, often, even in > 100MLOC code bases, there are only a handful of corner cases.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>I think I will give that solution a try, but it seems complex to implement and very limited in its ability to accept variations in the generated code.<br></div></div></div></div></blockquote><div><br></div></span><div>It seems to me that it allows you to be much more specific in addressing what you need.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div></div>My hope is still that there is a "keep the node in the ast" solution for clang that would allow me to rewrite such a statement based on the parameter type without caring about the exact statement within the brackets.<br></div></div></div></blockquote><div><br></div></span><div>But then the question is how can you be sure that it works afterwards? For that it seems like you'd need to know the exact information you would get when analyzing the unchanged code first (which is why I suggested that).</div><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><br></div>Greetings,<br></div>Marc<br><div><div><div><div><div><div><div><div><div><div><div><div><br></div></div></div></div></div></div></div></div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 9, 2014 at 2:27 PM, Manuel Klimek <span dir="ltr"><<a href="mailto:klimek@google.com" target="_blank">klimek@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I assume you cannot change the code generator?<br><br>Why can't you:<div>1. generate the code; parse it with the current version (having the 'int x')</div><div>2. find all 'int x's you wan to change to 'X x;'; also find all uses of them (including uses in array[x]); output all this information in some format </div><div>3. run over all those cases; now you can change 'int x' to 'X x' and 'array[x]' to 'myfunc(array, x)' at the same time</div><div>4. reap benefits; codebase is never in a non-parsing state</div><div><br></div><div>Cheers,</div><div>/Manuel</div><div><div><br><div class="gmail_quote">On Thu Oct 09 2014 at 2:10:51 PM Marc Greim <<a href="mailto:marc.greim@mytum.de" target="_blank">marc.greim@mytum.de</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">In this particular example "array[x]" was generated externally while I changed the declaration "int x;" to "X x;". That is the point where i need to patch the generated code in order to fix the missing operator[] error for type X and allow other simulation relevant operations.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 9, 2014 at 2:04 PM, Manuel Klimek <span dir="ltr"><<a href="mailto:klimek@google.com" target="_blank">klimek@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><span>On Thu Oct 09 2014 at 1:59:13 PM Marc Greim <<a href="mailto:marc.greim@mytum.de" target="_blank">marc.greim@mytum.de</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>In general I would also say that doing code transformation should only be done on valid code since one needs to know what actually happens. This particular problem is unfortunately rather specific. Sorry if the given example was not sufficient.<br><br></div>Maybe the problem becomes clearer when class X is defined as wrapper for int with special functionality. <br><br>The code that I try to transform is part of a hardware simulation written/generated in c++. The code is guaranteed to be valid except for the partial substitution of int variables by X variables. The resulting invalid code statements are only invalid in the sense that a parameter type is not right. Due to the complexity of the code it is not feasible/possible to predict where type X and where int is used for such operations. Since some operators (e.g. [] for pointers) cannot be overloaded with standard c++ code, errors will come up. For that and other reasons it is necessary to rewrite the code to use custom functions instead of the operator itself. This is where my problem with clang lies. Those nodes are removed from the ast due to missing operator[]/missing type conversion. But those are the nodes i need to preserve in order to run a matcher and transform the code as needed for the simulation. Again as noted before adding operator int() to class X is not a solution since that would create many ambiguity problems.<br><br><br></div>So to boil the problem further down: How can i forces clang to ignore wrong types that are passed to operators/functions and build the ast with such nodes?<br><br></div><div>Again, I suspect that adding built-in operators for those cases is the way to go, but I don't know how to iterate over all types and then create empty dummy functions for that.<br></div><div><br></div>I hope this describes my problem sufficiently.<br></div></blockquote><div><br></div></span><div>I still don't fully understand what the current situation is.</div><div>So you have code that calls array[x] with a class type x? How did you produce that code?</div><span><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 9, 2014 at 12:53 PM, Manuel Klimek <span dir="ltr"><<a href="mailto:klimek@google.com" target="_blank">klimek@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The general advise is to only do code transformations on valid code. I don't know enough about your problem to understand why that is not possible.<br><br><div class="gmail_quote"><div><div>On Tue Oct 07 2014 at 4:37:52 PM Marc Greim <<a href="mailto:marc.greim@mytum.de" target="_blank">marc.greim@mytum.de</a>> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr"><div><div><div><div><div>Hello,<br><br></div>I'm using clang to rewrite (generated) source code. Usually that can be done by running matchers on the ast in combination with a rewriter instance. <br><br>Unfortunately this fails in the case of operator[] where the index argument cannot be converted to a valid type (e.g. class X{} x; int * array; int i = array[x]; ). The corresponding ast nodes are missing since the "array[x]" statement has no valid representation.<br><br></div><div>How can I detect and rewrite the code in cases like above example? "array[x]" -> "someFunc(array,x)"<br><br></div><div>ExternalSemaSource::CorrectTypo dosen't get called in this case (maybe because all tokens are valid?) so that attempt failed.<br><br></div><div>Using ExternalSemaSource::LookupUnqualified also failed so far, because i haven't found a method to get a valid SourceRange for that code part. It would also require manual parsing of that code part which seem like a "dirty" solution to me.<br></div><div><br></div><div>The only idea that I have left is to declare builtin operators for any type with Sema::AddBuiltinCandidate, but that may result in many operator definitions. Also I have no idea how to iterate over all types and how to declare these functions. However this may be the best solution, because then matchers can be used to find and rewrite those code parts.<br><br></div><div>I would appreciate any help to find a solution.<br><br></div><div>Regards,<br></div><div>Marc<br><br></div><div><br></div><div>P.S. I am aware that adding operator int() to class X of above example would allow those statements but that is not an option, since X cannot be represented as int in my case and additional operations need to be performed; such an operator may also mess up other parts of the code.<br></div></div></div></div></div></div></div>
______________________________<u></u>_________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@cs.uiuc.edu" target="_blank">cfe-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/<u></u>mailman/listinfo/cfe-dev</a><br>
</blockquote></div>
</blockquote></div><br></div>
</blockquote></span></div>
</blockquote></div><br></div>
</blockquote></div>
</div></div></blockquote></div><br></div>
</blockquote></div></div></div>
</blockquote></div><br></div>