[cfe-dev] Is there a way to let the AST matcher run on multiple files in parallel?

[Han Wang]

Thanks! :) Let me see if I could generate some patch files then it would be
easier for the synchronization.

On Fri, Jul 31, 2015 at 10:17 AM, Manuel Klimek <klimek at google.com> wrote:

> That's why I wrote in my previous mail:
> "make sure your tool outputs the replacements that you want to apply in
> some format you can later slurp in and apply easily."
> The "apply" part is going to be single threaded.
>
> On Fri, Jul 31, 2015 at 9:30 AM Han Wang <wanghan02 at gmail.com> wrote:
>
>> Hi Manuel,
>>
>> I currently got some synchronization problem with this method. If a
>> header file is included by multiple source files, the write operations to
>> the header file are not synchronized. But if we use only 1 thread, it is
>> synchronized.
>>
>> Best regards,
>> Han
>>
>> On Wed, Jul 22, 2015 at 2:49 PM, Manuel Klimek <klimek at google.com> wrote:
>>
>>> Ok, so make sure your tool outputs the replacements that you want to
>>> apply in some format you can later slurp in and apply easily. Make the name
>>> of the output file depend on on the name of the translation unit you got at
>>> the command line.
>>>
>>> Then you can do:
>>> $ ls test1.cc test2.cc |xargs -n1 -P2 ./matchertool.out
>>>
>>> Unfortunately currently I don't think there's a simpler way than that.
>>>
>>> On Wed, Jul 22, 2015 at 2:42 PM Han Wang <wanghan02 at gmail.com> wrote:
>>>
>>>> Hi Manuel,
>>>>
>>>> That I have the compilation database already. :)
>>>>
>>>> Let's say we have a simple case with only 2 source files (test1.cc and
>>>> test2.cc). We also have the compilation database for these 2 files. And we
>>>> have the AST matcher tool up and running. Normally I'd run
>>>> ./matchertool.out test1.cc test2.cc to do the refactoring. To make the
>>>> analyzing of the 2 files running in parallel, what shall we do next?
>>>>
>>>> Thanks!
>>>> Best regards,
>>>> Han
>>>>
>>>> On Wed, Jul 22, 2015 at 2:29 PM, Manuel Klimek <klimek at google.com>
>>>> wrote:
>>>>
>>>>> Those are not 2 options. You'll want to have both :)
>>>>> You need the compilation database so the tool only needs the source
>>>>> file to run.
>>>>>
>>>>> On Wed, Jul 22, 2015 at 2:27 PM Han Wang <wanghan02 at gmail.com> wrote:
>>>>>
>>>>>> Hi Manuel,
>>>>>>
>>>>>> Thank you for your answer. The goal is to do refactoring to the
>>>>>> project and apply the replacements to the original files. For the option
>>>>>> (a), could you describe more specifically how to do it? Thanks. I checked
>>>>>> the run function of clang::tooling::ClangTool, but it's just a for loop
>>>>>> over all the source files.
>>>>>>
>>>>>> Best regards,
>>>>>> Han
>>>>>>
>>>>>> On Wed, Jul 22, 2015 at 12:33 PM, Manuel Klimek <klimek at google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> You can use
>>>>>>> a) a compilation database
>>>>>>> b) find . -name '*.cc' |xargs -n1 -P32
>>>>>>> depending on the output data you want to produce, you'll need to
>>>>>>> pass an additional flag to your tool to get the shard number...
>>>>>>>
>>>>>>> On Tue, Jul 21, 2015 at 6:20 PM Han Wang <wanghan02 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> Is there a way to let the AST matcher run on multiple files in
>>>>>>>> parallel? I'm working on Mac. I have some old windows code which may slow
>>>>>>>> down the compiling time (maybe for some other reasons I don't know
>>>>>>>> exactly). So this makes it necessary for me to analyze each translation
>>>>>>>> unit in parallel.
>>>>>>>>
>>>>>>>> At the moment I have only one mac which has 32 cores. I can live
>>>>>>>> with that if it could be fully parallel.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>> Best regards,
>>>>>>>> Han
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> cfe-dev mailing list
>>>>>>>> cfe-dev at cs.uiuc.edu
>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150731/621b8bcd/attachment.html>