[cfe-dev] libclang cancellation of long running tasks

[John Sully]

> The problem is that most of the time is spent throughout various
functions of Sema in clang, so the only way I see is to have every function
in Sema check for abortion, which sounds like it would horribly clutter the
code...

I don't think it needs to be that extreme.  Once every 100ms would be
sufficient for my purposes which would drastically reduce the number of
checks needed.  My own profiles are also showing a fair amount of time in
the Lex code as well.  I'm still learning the code but I suspect it would
be feasible to check in-between major phases.

On Tue, Jan 13, 2015 at 2:27 AM, Manuel Klimek <klimek at google.com> wrote:

> On Tue Jan 13 2015 at 11:18:38 AM John Sully <john at csquare.ca> wrote:
>
>> I'm pretty confident there is no current way to do it, so this week I'll
>> start on my own implementation.  For reasons I don't quite understand
>> libclang likes to spawn jobs onto seperate threads (and therefore ignores
>> my thread priorities...) which will make this somewhat more difficult.
>>
>> I think the sanest way is to pass a bool FCheckAbort() style function
>> pointer as an optional parameter to jobs which may run a long time.  I'll
>> have to do some work to marshal this across thread boundaries.  Either that
>> or remove the spawned thread entirely.
>>
>
> The problem is that most of the time is spent throughout various functions
> of Sema in clang, so the only way I see is to have every function in Sema
> check for abortion, which sounds like it would horribly clutter the code...
>
>
>>
>>
>> On Mon, Jan 12, 2015 at 11:41 PM, Manuel Klimek <klimek at google.com>
>> wrote:
>>
>>> I'll add my +1 to it being unfortunate that libclang operations cannot
>>> be canceled (living in a code base where some TUs take > 30 seconds to
>>> parse).
>>>
>>>
>>> On Tue Jan 13 2015 at 3:38:51 AM John Sully <john at csquare.ca> wrote:
>>>
>>>> Hi Jonathan,
>>>>
>>>> I don't understand how std::future::wait_for would allow me to cancel a
>>>> running libclang operation.
>>>>
>>>> I'm not asking how to do this asynchronously - I already have that
>>>> working.  I need a way to stop performing work when the results become
>>>> obsolete. I could always shutdown the thread its on but that would leave
>>>> things in an undefined state.
>>>>
>>>>
>>>> On Mon, Jan 12, 2015 at 6:04 PM, Jonathan Roelofs <
>>>> jonathan at codesourcery.com> wrote:
>>>>
>>>>> John,
>>>>>
>>>>> It should be fairly straightforward for you to implement this behavior
>>>>> in your application using std::future::wait_for.
>>>>>
>>>>>
>>>>> Jon
>>>>>
>>>>> On 1/12/15 6:33 PM, John Sully wrote:
>>>>>
>>>>>> Yes it already runs on a separate thread; however when its known the
>>>>>> result is no longer necessary (for example the user changed the cursor
>>>>>> location - negating the need for autocomplete at the previous
>>>>>> location)
>>>>>> the separate thread still runs consuming resources.
>>>>>>
>>>>>> On slower devices like laptops these operations can take a long time.
>>>>>> Even if we are willing to accept the waste of cycles (and battery
>>>>>> life!), the processor has a limited number of cores and responsiveness
>>>>>> will be poor if too many "zombie" jobs are still running.
>>>>>>
>>>>>> I'm willing to add this functionality and submit it upstream if
>>>>>> necessary, but prior to doing so I wanted to reach out to current
>>>>>> developers.  It would be unfortunate to invest significant effort only
>>>>>> to find my patch can't be accepted for an avoidable reason.
>>>>>>
>>>>>>
>>>>>> On Mon, Jan 12, 2015 at 4:09 PM, Sean Silva <chisophugis at gmail.com
>>>>>> <mailto:chisophugis at gmail.com>> wrote:
>>>>>>
>>>>>>     Have you tried running libclang on a separate thread?
>>>>>>
>>>>>>     On Sun, Jan 11, 2015 at 7:59 PM, John Sully <john at csquare.ca
>>>>>>     <mailto:john at csquare.ca>> wrote:
>>>>>>
>>>>>>         I'm in the process of writing an editor based upon libclang,
>>>>>> and
>>>>>>         a major issue I'm hitting is the inability to cancel long
>>>>>>         running tasks.  The primary tasks are parsing and creation of
>>>>>>         auto-complete results.
>>>>>>
>>>>>>         Is there already functionality to do this that I've missed?
>>>>>>
>>>>>>         If not, I would like to implement this as its a requirement to
>>>>>>         get good responsiveness out of my editor.  Has there been any
>>>>>>         prior discussions on how this should be implemented?
>>>>>>
>>>>>>         _______________________________________________
>>>>>>         cfe-dev mailing list
>>>>>>         cfe-dev at cs.uiuc.edu <mailto:cfe-dev at cs.uiuc.edu>
>>>>>>         http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> cfe-dev mailing list
>>>>>> cfe-dev at cs.uiuc.edu
>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>>>
>>>>>>
>>>>> --
>>>>> Jon Roelofs
>>>>> jonathan at codesourcery.com
>>>>> CodeSourcery / Mentor Embedded
>>>>>
>>>>
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150113/b8ad1c9a/attachment.html>