[cfe-dev] Source to source transformations questions in branches/tooling

[Manuel Klimek]

On Thu, Jun 28, 2012 at 1:31 AM, Philip Dunstan <phil at philipdunstan.com>wrote:

>
> Manuel Klimek wrote
> >
> > On Sun, Jun 24, 2012 at 4:53 PM, Philip Dunstan <phil@>wrote:
> >
> >> Replace function parameters that specify a shared_ptr instance by
> >> value with shared_ptr by const reference.
> >>
> >> Should I be changing the type from shared_ptr<X> to const shared_ptr<X>&
> >> or
> >> is there a way to do this by modifying the AST to make the parameter
> >> const
> >> and by reference and then re-emit the source?
> >>
> >
> > If you can match the location of the shared_ptr and the type, you can
> emit
> > a replacement to "const X&" (no need to go through the AST here). Is
> there
> > a part of the matching that's problematic?
> >
>
> Nothing problematic. It just feels like I might lose something with a
> simple
> string replacement. I am not sure what might so wrong, so I might just be
> being a little over cautious here.
>

We've been doing simple string replacements for large scale code
transformations on google's repo for about 1.5 years now :)


> Manuel Klimek wrote
> >
> >> It appears that I need to specify all of the system c++ include files
> >> when
> >> I run the tool on a file, but I don't need to provide them when I
> compile
> >> the with the clang compiler that I have built from the tooling branch.
> Is
> >> this expected behavior?
> >
> > Hm, this should work in principle. Can you add '-v' output to both the
> > tool
> > run and the clang run from your self-built clang compiler and attach
> them?
> >
>
> I've added the outputs with '-v'. The first is clang built from the tooling
> branch, the second is my tool built against the same tooling branch. The
> only difference looks like the include path from the clang build directory,
> and explicitly specifying this has fixed the errors from the tool. Who
> should be responsible for adding this path to the tools header search list?
>

Hah, you're cheating :)
Clang tools need the clang builtin headers just like clang. So the best way
to run them is to put them into the same binary directory where clang is
located.

Or you convince Chandler on IRC that we want to have the builtin headers
compiled into the binary for tools ;)

Cheers,
/Manuel


>
>
> > /home/phil/Development/tooling/build/Debug+Asserts/bin/clang++ -c
> > testdata.cpp -v -std=c++11
> clang version 3.2 (branches/tooling 158516)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
>  "/home/phil/Development/tooling/build/Debug+Asserts/bin/clang" -cc1
> -triple
> x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free
> -main-file-name
> testdata.cpp -mrelocation-model static -mdisable-fp-elim -fmath-errno
> -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64
> -target-linker-version 2.22 -momit-leaf-frame-pointer -v -coverage-file
> testdata.o -resource-dir
> /home/phil/Development/tooling/build/Debug+Asserts/bin/../lib/clang/3.2
> -fmodule-cache-path /var/tmp/clang-module-cache -internal-isystem
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6
> -internal-isystem
>
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/x86_64-linux-gnu
> -internal-isystem
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/backward
> -internal-isystem /usr/local/include -internal-isystem
>
> /home/phil/Development/tooling/build/Debug+Asserts/bin/../lib/clang/3.2/include
> -internal-externc-isystem /usr/include/x86_64-linux-gnu
> -internal-externc-isystem /include -internal-externc-isystem /usr/include
> -std=c++11 -fdeprecated-macro -fdebug-compilation-dir
> /home/phil/Development/clang_tools/add-override-specifier -ferror-limit 19
> -fmessage-length 138 -mstackrealign -fgnu-runtime -fobjc-runtime-has-arc
> -fobjc-runtime-has-weak -fobjc-fragile-abi -fcxx-exceptions -fexceptions
> -fdiagnostics-show-option -fcolor-diagnostics -o testdata.o -x c++
> testdata.cpp
> clang -cc1 version 3.2 based upon LLVM 3.2svn default target
> x86_64-unknown-linux-gnu
> ignoring nonexistent directory "/include"
> #include "..." search starts here:
> #include <...> search starts here:
>  /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6
>
>
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/x86_64-linux-gnu
>  /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/backward
>  /usr/local/include
>
>
> /home/phil/Development/tooling/build/Debug+Asserts/bin/../lib/clang/3.2/include
>  /usr/include/x86_64-linux-gnu
>  /usr/include
> End of search list.
>
>
> > ./add-override-specifier testdata.cpp -- -v -std=c++11
> Processing:
> /home/phil/Development/clang_tools/add-override-specifier/testdata.cpp.
> clang version 3.2 (branches/tooling 158516)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
> clang Invocation:
>
>
> "/home/phil/Development/clang_tools/add-override-specifier/add-override-specifier"
> "-cc1" "-triple" "x86_64-unknown-linux-gnu" "-fsyntax-only" "-disable-free"
> "-main-file-name" "testdata.cpp" "-mrelocation-model" "static"
> "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases"
> "-munwind-tables" "-target-cpu" "x86-64" "-target-linker-version" "2.22"
> "-momit-leaf-frame-pointer" "-v" "-resource-dir"
>
> "/home/phil/Development/clang_tools/add-override-specifier/../lib/clang/3.2"
> "-fmodule-cache-path" "/var/tmp/clang-module-cache" "-internal-isystem"
> "/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6"
> "-internal-isystem"
>
> "/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/x86_64-linux-gnu"
> "-internal-isystem"
> "/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/backward"
> "-internal-isystem" "/usr/local/include" "-internal-isystem"
>
> "/home/phil/Development/clang_tools/add-override-specifier/../lib/clang/3.2/include"
> "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu"
> "-internal-externc-isystem" "/include" "-internal-externc-isystem"
> "/usr/include" "-std=c++11" "-fdeprecated-macro" "-fdebug-compilation-dir"
> "/home/phil/Development/clang_tools/add-override-specifier" "-ferror-limit"
> "19" "-fmessage-length" "138" "-mstackrealign" "-fgnu-runtime"
> "-fobjc-runtime-has-arc" "-fobjc-runtime-has-weak" "-fobjc-fragile-abi"
> "-fcxx-exceptions" "-fexceptions" "-fdiagnostics-show-option"
> "-fcolor-diagnostics" "-x" "c++"
> "/home/phil/Development/clang_tools/add-override-specifier/testdata.cpp"
>
> clang -cc1 version 3.2 based upon LLVM 3.2svn default target
> x86_64-unknown-linux-gnu
> ignoring nonexistent directory
>
> "/home/phil/Development/clang_tools/add-override-specifier/../lib/clang/3.2/include"
> ignoring nonexistent directory "/include"
> #include "..." search starts here:
> #include <...> search starts here:
>  /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6
>
>
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/x86_64-linux-gnu
>  /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/backward
>  /usr/local/include
>  /usr/include/x86_64-linux-gnu
>  /usr/include
> End of search list.
> In file included from
> /home/phil/Development/clang_tools/add-override-specifier/testdata.cpp:1:
> In file included from
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/iostream:39:
> In file included from
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/ostream:39:
> In file included from
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/ios:38:
> In file included from
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/iosfwd:41:
> In file included from
>
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/bits/postypes.h:41:
> In file included from
> /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/cwchar:45:
> /usr/include/wchar.h:40:11: fatal error: 'stdarg.h' file not found
> # include <stdarg.h>
>          ^
> 1 error generated.
> Error while processing
> /home/phil/Development/clang_tools/add-override-specifier/testdata.cpp.
>
>
>
>
> Manuel Klimek wrote
> >
> > A different solution would be to use the CompilationDatabase. If your
> > project uses cmake, it works out of the box; if not, the easiest thing is
> > to create a compilation database. See the docs here:
> > http://clang.llvm.org/docs/JSONCompilationDatabase.html
> >
> I will definitely look into this. We use scons rather than cmake, but it
> should no problem extracting this information.
>
>
>
> Manuel Klimek wrote
> >
> > I simply do match on the path / name of the files for that. Usually
> > project
> > files have easily detectable paths.
> > Also note that what I do is key whether I want to replace something on
> the
> > file of the original declaration of the 'thing' I want to replace, so
> that
> > you don't get inconsistent states.
> >
> Interesting. That is good to know.
>
>
> Manuel Klimek wrote
> >
> > That's what the refactoring support in the tooling library is for:
> >
> http://llvm.org/viewvc/llvm-project/cfe/branches/tooling/include/clang/Tooling/Refactoring.h?view=markup
> >
> > The idea is that you run over all TUs you want to refactor in the same
> > process. We're working on getting this multi-threaded so that it scales
> > better.
> >
> > Another possibility is to output simple diffs from your tool, and use a
> > little python script to slurp in the changes, unique them and apply them
> > (that's what we currently do for really large scale MapReduce style
> > refactorings).
> >
> Thanks, I will definitely look into that
>
> Phil
> --
> Philip Dunstan
> phil at philipdunstan.com
> www.philipdunstan.com
>
>
>
> --
> View this message in context:
> http://clang-developers.42468.n3.nabble.com/Source-to-source-transformations-questions-in-branches-tooling-tp4024920p4025005.html
> Sent from the Clang Developers mailing list archive at Nabble.com.
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20120628/dabffa6d/attachment.html>