[cfe-dev] [clangd] Worker threads stack size

[Ilya Biryukov via cfe-dev]

llvm::execute_on_thread waits for completion of the operation, clangd
spawns long-living threads managing requests to the AST and short-lived
threads for async processing of requests.

Both are asynchronous and we only join them when clangd exits.
We might have something similar to llvm:: execute_on_thread, but that would
mean reimplementing a big chunk of threading lib from STL, I think it's
exactly what Dmitry proposed as an option.

Alex Lorenz <aleksei_lorenz at apple.com> schrieb am Fr., 17. Aug. 2018, 20:05:

> I think the best solution is to use `llvm::llvm_execute_on_thread` instead
> of `std::thread`.
>
> Cheers,
> Alex
>
> On Aug 17, 2018, at 7:12 AM, Roman Popov via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
> In case MacOS LibC++ is implemented on top of pthreads, than you can can
> use pthreads API to set default stack size. This will probably be miminal
> possible platform-specific workaround.
>
> пт, 17 авг. 2018 г. в 6:27, Ilya Biryukov via cfe-dev <
> cfe-dev at lists.llvm.org>:
>
>> Would be nice to get around this limitation in the standard library. I
>> don't think we can do it in a standard-compliant way, though.
>> There's `llvm::llvm_execute_on_thread`, but it is an OS-specific
>> implementation.
>>
>> Are there any ways to set default stack sizes for the newly created
>> threads? Even if it's platform-specific code, it should be pretty localized
>> and we could reuse the STL implementation of threads.
>> I don't know of reliable APIs to do that, unfortunately.
>>
>> On Fri, Aug 17, 2018 at 2:43 PM Eric Liu <ioeric at google.com> wrote:
>>
>>> +clangd-dev at lists.llvm.org <clangd-dev at lists.llvm.org> +Ilya Biryukov
>>> <ibiryukov at google.com>
>>>
>>> On Fri, Aug 17, 2018 at 2:41 PM Dmitry.Kozhevnikov at jetbrains.com via
>>> cfe-dev <cfe-dev at lists.llvm.org> wrote:
>>>
>>>> Hello!
>>>>
>>>> clangd uses std::thread for spawning worker threads to parse files (see
>>>> AsyncTaskRunner::runAsync in tools/clang/tools/extra/clangd/Threading.cpp).
>>>> Unfortunately, std::thread lacks the ability to customize the stack size,
>>>> and the default one might be insufficient for parsing complex source files.
>>>> For example, on macOS, the default stack size for non-main threads is 512K,
>>>> so the following code causes a stack overflow (on my machine) with clangd,
>>>> while being way below the default constexpr evaluation depth threshold:
>>>>
>>>>         constexpr int foo(int x) {
>>>>             return x == 1 ? 1 : foo(x - 1) + 1;
>>>>         }
>>>>         static_assert(foo(50) > 0, "");
>>>>
>>>> This also can be observed on some real-world projects.
>>>>
>>>> Do I understand correctly that the only way to fix it would be to
>>>> abandon std::thread and reimplement the platform-specific thread spawning
>>>> (using pthreads/WinAPI), or is there another solution available in
>>>> LLVM/clang codebase?
>>>>
>>>> Best regards,
>>>> Dmitry Kozhevnikov._______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>>
>>
>> --
>> Regards,
>> Ilya Biryukov
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180817/5496dce1/attachment.html>