[cfe-dev] Concurrent AST traversal with libTooling

[Alp Toker]

On 27/01/2014 13:33, Manuel Klimek wrote:
> On Thu, Jan 16, 2014 at 1:55 AM, Marc Jessome <marc.jessome at gmail.com 
> <mailto:marc.jessome at gmail.com>> wrote:
>
>     Hello,
>
>     I'm using libTooling to parse and traverse the ASTs of full
>     compilation databases using a RecursiveASTVisitor. I have attached a
>     small example of how my tool is structured (a modification of
>     http://clang.llvm.org/docs/RAVFrontendAction.html to use a compilation
>     database).
>
>     I would like to be able to parse each TU and traverse the AST on its
>     own thread so that I can process multiple TUs concurrently.
>
>     Would this be possible with libTooling and tooling::ClangTool as
>     currently implemented and used in my example? Give the implementation
>     of newFrontendActionFactory<> I feel like it likely isn't.
>
>     If not, the only way that I can currently see this being possible is
>     by spawning up multiple threads that would each call
>     tooling::runToolOnCodeWithArgs(), and handling the compilation
>     database manually. 
>
>
> The other way would be to use one process per TU, and have an on-disk 
> intermediate output format. That's what we do, as it nicely scales to 
> many machines.
>
>     Would this be safe to do with the way clang and
>     libTooling is currently implemented?
>
>
> This is somewhat doable if you are sure that your repository will 
> always stay in the same working directory. Unfortunately clang 
> currently chdirs, and the correct way to resolve this would be by 
> refactoring the filesystem/path libs to allow to get directories, and 
> then resolve paths relative to those directories, which is a bunch of 
> work.

Manuel, could you put together for a PR for the chdir describing what 
relies on the cwd?

Perhaps we can figure out a light way to simulate the behaviour without 
major surgery.

Alp.


>
> Cheers,
> /Manuel
>
>
>     Thanks,
>       Marc
>
>     _______________________________________________
>     cfe-dev mailing list
>     cfe-dev at cs.uiuc.edu <mailto:cfe-dev at cs.uiuc.edu>
>     http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

-- 
http://www.nuanti.com
the browser experts