[cfe-dev] Source-to-Source transformation (as a plugin?) - state of the art

[rNoz via cfe-dev]

Hi everybody,

As a beginner with clang I am struggling to support source-to-source
transformation in one step with clang.

How is the state of the art in source-to-source transformation with clang?

I followed almost every resource on the Web and I can achieve to do
source rewrites (Rewriter) via a clang plugin, but the final binary is
not updated (the CodeGen is the main activity, and it is compiled
regardless of what I have modified in my plugin, even using
AddBeforeMainAction in the getActionType).

I have seen some documents regarding libTooling and how to create an
independent program that uses clang as library, but my purpose it is to
create a plugin (FrontendPluginRegistry::Add<>, something "easy" to plug
to a non-custom clang binary) and achieve source-to-source modifications
(transparently to the users, avoiding overwriting their source files).

Being a bit pedantic, in case it is not clear: I need something like a
"plugin" to extend clang in an easy way. I need something that is
"integrated" in the compiling process. Why? because I need to modify the
source code during the compilation phase, inject new code, modify the
source code from the user in one step (I don't want to create a tool to
parse the user source code and then compile the output files). Also, I
would like to distribute my code (plugin) to allow users to use it by
themselves.

It is mandatory that it is during the compilation phase of clang (clang
$FLAGS $PLUGIN $ETC -o program source_files...).


**Use cases I would like to try:**
- 1. I would like to transform the source code in a really fast and
confident way, even before the lexer. I don't know if it is possible.
Like having a stream of text and transforming, previous to the
preprocessor/lexer. I know it can be dangerous, but at least to know if
I can and **how to proceed**.
- 2. I would like to transform specific parts of the source code (can be
with the Rewriter, as I have achieved). For that, I use the ASTConsumer
and Visitor pattern. Then, I would like to rewrite, inject text, etc.
**So far I know how to do it, but not how to preserve for the CodeGen
(main action) and create the final binary with it (reparsing again the
source code and going through every phase).**
- 3. I would like to create AST Nodes. Although I have read it is not
the best option, I would like to be able to add new nodes, statements,
etc. *I didn't try yet this phase, but any guess?*


Regards, rNoz

Some threads, repositories or blogs that I have reviewed and tested. In
some of them there are answers that don't work:

https://github.com/eliben/llvm-clang-samples/blob/master/src_clang/rewritersample.cpp 

https://github.com/fmtlib/fmt 
https://github.com/amorri40/ClangFunctionInstrumentation/blob/master/MyRecursiveASTVisitor.cpp 

https://github.com/RadeonOpenCompute/HCC_syscall/blob/e3fa6c24a6525c66006218ee33dbd0bc049cb511/lib/clang-plugins/StmtRewriter.cpp 

https://github.com/CPFL/gloop/tree/d30bdd267c2453f71167efca79afbfb8771b8a28 

https://jonasdevlieghere.com/understanding-the-clang-ast/ 
https://github.com/sinelaw/elfs-clang-plugins/tree/master/large_assignment 
https://github.com/loarabia/Clang-tutorial/blob/master/CIrewriter.cpp 
https://github.com/loarabia/Clang-tutorial/wiki/TutorialOrig http://clang-developers.42468.n3.nabble.com/RFC-Add-custom-built-in-functions-using-a-plugin-tt4052587.html#none 

https://github.com/eliben/llvm-clang-samples/blob/master/src_clang/matchers_rewriter.cpp 

http://clang-developers.42468.n3.nabble.com/Adding-nodes-to-Clang-s-AST-td4054800.html
https://github.com/vgvassilev/clad/tree/master/tools 
**http://clang-developers.42468.n3.nabble.com/How-to-make-clang-compile-the-data-from-the-RewriteBuffer-td4041049.html**

https://github.com/sampsyo/quala/blob/master/TypeAnnotations.h 
**http://lists.llvm.org/pipermail/cfe-dev/2016-November/051668.html**
https://stackoverflow.com/questions/32941882/changing-the-source-code-with-a-clang-plugin

I mark the "most important" in bold. The second one just hits the knot,
I am in the same "state" as him.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20171017/cf5fb7b4/attachment.html>