[cfe-dev] libclang cancellation of long running tasks

Tue Jan 13 03:17:50 PST 2015

On Tue Jan 13 2015 at 12:13:53 PM John Sully <john at csquare.ca> wrote:

> > The problem is that most of the time is spent throughout various
> functions of Sema in clang, so the only way I see is to have every function
> in Sema check for abortion, which sounds like it would horribly clutter the
> code...
>
> I don't think it needs to be that extreme.  Once every 100ms would be
> sufficient for my purposes which would drastically reduce the number of
> checks needed.  My own profiles are also showing a fair amount of time in
> the Lex code as well.  I'm still learning the code but I suspect it would
> be feasible to check in-between major phases.
>

I suspect it won't, but feel free to give it a try - for really long TUs we
see all of the time spent in Sema trying to parse C++, and it's basically a
flat profile.

> On Tue, Jan 13, 2015 at 2:27 AM, Manuel Klimek <klimek at google.com> wrote:
>
>> On Tue Jan 13 2015 at 11:18:38 AM John Sully <john at csquare.ca> wrote:
>>
>>> I'm pretty confident there is no current way to do it, so this week I'll
>>> start on my own implementation.  For reasons I don't quite understand
>>> libclang likes to spawn jobs onto seperate threads (and therefore ignores
>>> my thread priorities...) which will make this somewhat more difficult.
>>>
>>> I think the sanest way is to pass a bool FCheckAbort() style function
>>> pointer as an optional parameter to jobs which may run a long time.  I'll
>>> have to do some work to marshal this across thread boundaries.  Either that
>>> or remove the spawned thread entirely.
>>>
>>
>> The problem is that most of the time is spent throughout various
>> functions of Sema in clang, so the only way I see is to have every function
>> in Sema check for abortion, which sounds like it would horribly clutter the
>> code...
>>
>>
>>>
>>>
>>> On Mon, Jan 12, 2015 at 11:41 PM, Manuel Klimek <klimek at google.com>
>>> wrote:
>>>
>>>> I'll add my +1 to it being unfortunate that libclang operations cannot
>>>> be canceled (living in a code base where some TUs take > 30 seconds to
>>>> parse).
>>>>
>>>>
>>>> On Tue Jan 13 2015 at 3:38:51 AM John Sully <john at csquare.ca> wrote:
>>>>
>>>>> Hi Jonathan,
>>>>>
>>>>> I don't understand how std::future::wait_for would allow me to cancel
>>>>> a running libclang operation.
>>>>>
>>>>> I'm not asking how to do this asynchronously - I already have that
>>>>> working.  I need a way to stop performing work when the results become
>>>>> obsolete. I could always shutdown the thread its on but that would leave
>>>>> things in an undefined state.
>>>>>
>>>>>
>>>>> On Mon, Jan 12, 2015 at 6:04 PM, Jonathan Roelofs <
>>>>> jonathan at codesourcery.com> wrote:
>>>>>
>>>>>> John,
>>>>>>
>>>>>> It should be fairly straightforward for you to implement this
>>>>>> behavior in your application using std::future::wait_for.
>>>>>>
>>>>>>
>>>>>> Jon
>>>>>>
>>>>>> On 1/12/15 6:33 PM, John Sully wrote:
>>>>>>
>>>>>>> Yes it already runs on a separate thread; however when its known the
>>>>>>> result is no longer necessary (for example the user changed the
>>>>>>> cursor
>>>>>>> location - negating the need for autocomplete at the previous
>>>>>>> location)
>>>>>>> the separate thread still runs consuming resources.
>>>>>>>
>>>>>>> On slower devices like laptops these operations can take a long time.
>>>>>>> Even if we are willing to accept the waste of cycles (and battery
>>>>>>> life!), the processor has a limited number of cores and
>>>>>>> responsiveness
>>>>>>> will be poor if too many "zombie" jobs are still running.
>>>>>>>
>>>>>>> I'm willing to add this functionality and submit it upstream if
>>>>>>> necessary, but prior to doing so I wanted to reach out to current
>>>>>>> developers.  It would be unfortunate to invest significant effort
>>>>>>> only
>>>>>>> to find my patch can't be accepted for an avoidable reason.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jan 12, 2015 at 4:09 PM, Sean Silva <chisophugis at gmail.com
>>>>>>> <mailto:chisophugis at gmail.com>> wrote:
>>>>>>>
>>>>>>>     Have you tried running libclang on a separate thread?
>>>>>>>
>>>>>>>     On Sun, Jan 11, 2015 at 7:59 PM, John Sully <john at csquare.ca
>>>>>>>     <mailto:john at csquare.ca>> wrote:
>>>>>>>
>>>>>>>         I'm in the process of writing an editor based upon libclang,
>>>>>>> and
>>>>>>>         a major issue I'm hitting is the inability to cancel long
>>>>>>>         running tasks.  The primary tasks are parsing and creation of
>>>>>>>         auto-complete results.
>>>>>>>
>>>>>>>         Is there already functionality to do this that I've missed?
>>>>>>>
>>>>>>>         If not, I would like to implement this as its a requirement
>>>>>>> to
>>>>>>>         get good responsiveness out of my editor.  Has there been any
>>>>>>>         prior discussions on how this should be implemented?
>>>>>>>
>>>>>>>         _______________________________________________
>>>>>>>         cfe-dev mailing list
>>>>>>>         cfe-dev at cs.uiuc.edu <mailto:cfe-dev at cs.uiuc.edu>
>>>>>>>         http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> cfe-dev mailing list
>>>>>>> cfe-dev at cs.uiuc.edu
>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Jon Roelofs
>>>>>> jonathan at codesourcery.com
>>>>>> CodeSourcery / Mentor Embedded
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> cfe-dev mailing list
>>>>> cfe-dev at cs.uiuc.edu
>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150113/1b6a3c90/attachment.html>