[cfe-dev] Concurrent AST traversal with libTooling

[Manuel Klimek]

On Thu, Jan 16, 2014 at 1:55 AM, Marc Jessome <marc.jessome at gmail.com>wrote:

> Hello,
>
> I'm using libTooling to parse and traverse the ASTs of full
> compilation databases using a RecursiveASTVisitor. I have attached a
> small example of how my tool is structured (a modification of
> http://clang.llvm.org/docs/RAVFrontendAction.html to use a compilation
> database).
>
> I would like to be able to parse each TU and traverse the AST on its
> own thread so that I can process multiple TUs concurrently.
>
> Would this be possible with libTooling and tooling::ClangTool as
> currently implemented and used in my example? Give the implementation
> of newFrontendActionFactory<> I feel like it likely isn't.
>
> If not, the only way that I can currently see this being possible is
> by spawning up multiple threads that would each call
> tooling::runToolOnCodeWithArgs(), and handling the compilation
> database manually.


The other way would be to use one process per TU, and have an on-disk
intermediate output format. That's what we do, as it nicely scales to many
machines.


> Would this be safe to do with the way clang and
> libTooling is currently implemented?
>

This is somewhat doable if you are sure that your repository will always
stay in the same working directory. Unfortunately clang currently chdirs,
and the correct way to resolve this would be by refactoring the
filesystem/path libs to allow to get directories, and then resolve paths
relative to those directories, which is a bunch of work.

Cheers,
/Manuel


>
> Thanks,
>   Marc
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20140127/352d3d85/attachment.html>