[cfe-dev] Is there a way to let the AST matcher run on multiple files in parallel?

[Manuel Klimek]

That's why I wrote in my previous mail:
"make sure your tool outputs the replacements that you want to apply in
some format you can later slurp in and apply easily."
The "apply" part is going to be single threaded.

On Fri, Jul 31, 2015 at 9:30 AM Han Wang <wanghan02 at gmail.com> wrote:

> Hi Manuel,
>
> I currently got some synchronization problem with this method. If a header
> file is included by multiple source files, the write operations to the
> header file are not synchronized. But if we use only 1 thread, it is
> synchronized.
>
> Best regards,
> Han
>
> On Wed, Jul 22, 2015 at 2:49 PM, Manuel Klimek <klimek at google.com> wrote:
>
>> Ok, so make sure your tool outputs the replacements that you want to
>> apply in some format you can later slurp in and apply easily. Make the name
>> of the output file depend on on the name of the translation unit you got at
>> the command line.
>>
>> Then you can do:
>> $ ls test1.cc test2.cc |xargs -n1 -P2 ./matchertool.out
>>
>> Unfortunately currently I don't think there's a simpler way than that.
>>
>> On Wed, Jul 22, 2015 at 2:42 PM Han Wang <wanghan02 at gmail.com> wrote:
>>
>>> Hi Manuel,
>>>
>>> That I have the compilation database already. :)
>>>
>>> Let's say we have a simple case with only 2 source files (test1.cc and
>>> test2.cc). We also have the compilation database for these 2 files. And we
>>> have the AST matcher tool up and running. Normally I'd run
>>> ./matchertool.out test1.cc test2.cc to do the refactoring. To make the
>>> analyzing of the 2 files running in parallel, what shall we do next?
>>>
>>> Thanks!
>>> Best regards,
>>> Han
>>>
>>> On Wed, Jul 22, 2015 at 2:29 PM, Manuel Klimek <klimek at google.com>
>>> wrote:
>>>
>>>> Those are not 2 options. You'll want to have both :)
>>>> You need the compilation database so the tool only needs the source
>>>> file to run.
>>>>
>>>> On Wed, Jul 22, 2015 at 2:27 PM Han Wang <wanghan02 at gmail.com> wrote:
>>>>
>>>>> Hi Manuel,
>>>>>
>>>>> Thank you for your answer. The goal is to do refactoring to the
>>>>> project and apply the replacements to the original files. For the option
>>>>> (a), could you describe more specifically how to do it? Thanks. I checked
>>>>> the run function of clang::tooling::ClangTool, but it's just a for loop
>>>>> over all the source files.
>>>>>
>>>>> Best regards,
>>>>> Han
>>>>>
>>>>> On Wed, Jul 22, 2015 at 12:33 PM, Manuel Klimek <klimek at google.com>
>>>>> wrote:
>>>>>
>>>>>> You can use
>>>>>> a) a compilation database
>>>>>> b) find . -name '*.cc' |xargs -n1 -P32
>>>>>> depending on the output data you want to produce, you'll need to pass
>>>>>> an additional flag to your tool to get the shard number...
>>>>>>
>>>>>> On Tue, Jul 21, 2015 at 6:20 PM Han Wang <wanghan02 at gmail.com> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Is there a way to let the AST matcher run on multiple files in
>>>>>>> parallel? I'm working on Mac. I have some old windows code which may slow
>>>>>>> down the compiling time (maybe for some other reasons I don't know
>>>>>>> exactly). So this makes it necessary for me to analyze each translation
>>>>>>> unit in parallel.
>>>>>>>
>>>>>>> At the moment I have only one mac which has 32 cores. I can live
>>>>>>> with that if it could be fully parallel.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Best regards,
>>>>>>> Han
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> cfe-dev mailing list
>>>>>>> cfe-dev at cs.uiuc.edu
>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>>>
>>>>>>
>>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150731/de03f51d/attachment.html>