[cfe-dev] clang crashes when running in VM

[Csaba Raduly via cfe-dev]

For N=1, you can limit the number of parallel linker processes by
making the linker command

flock /tmp/linker.lock gcc

instead of gcc (or whatever you use to link)

Then the rest of the build can go in parallel.


On Wed, Jul 26, 2017 at 8:34 PM, George Burgess IV via cfe-dev
<cfe-dev at lists.llvm.org> wrote:
>> I guess it might be due to parallel linking, the memory usage would skyrocket?
>
> Yup, that would do it. :)
>
>> Is there a graceful way to avoid this?
>
> If your project uses CMake, you might be able to have it generate
> files for `ninja` instead of Makefiles. Ninja has job pools just for
> cases like this: https://ninja-build.org/manual.html#ref_pool (and
> I've heard many LLVM devs talk about how using `ninja` instead of
> `make` made incremental rebuilds of LLVM/clang faster for them, so...)
>
> If you're stuck with Makefiles, though, I don't know of a great way to
> limit to N parallel link jobs. If you don't mind linking being 100%
> serial, you may find the (nonstandard, apparently) .NOTPARALLEL
> target useful.
>
> Good luck!
> George
>
> On Wed, Jul 26, 2017 at 10:17 AM, chen hsu <chenxuz at outlook.com> wrote:
>> Thanks for replying, George.
>>
>> I allocated 16GB to the VM, as well as 4GB swap. Our
>> executable/shared-library are pretty big, usually 500MB to 2GB in size with
>> debug symbols, and we use "make -j 12" for the 6-core (12 hyperthread) CPU
>> to be fully used. I guess it might be due to parallel linking, the memory
>> usage would skyrocket? Is there a graceful way to avoid this?
>>
>> ________________________________
>> From: George Burgess IV <george.burgess.iv at gmail.com>
>> Sent: Wednesday, July 26, 2017 2:20:21 AM
>> To: chen hsu
>> Cc: Clang Dev
>> Subject: Re: [cfe-dev] clang crashes when running in VM
>>
>> Hello!
>>
>> "error: unable to execute command: Killed" sounds a lot like what I
>> get when clang gets killed for eating all of my system's memory. Are
>> you sure that your VM has enough RAM to handle linking whatever it is
>> that you're linking?
>>
>> Thanks,
>> George
>>
>> On Tue, Jul 25, 2017 at 5:24 PM, chen hsu via cfe-dev
>> <cfe-dev at lists.llvm.org> wrote:
>>> Hi,
>>>
>>>
>>> we have clang 3.6.2 on ubuntu 16.04 to build our project and it works well
>>> on physical machines. but when running in VM (HyperV on Windows Server
>>> 2012R2), it crashes quite often in linking. Is this a known issue? It
>>> seems
>>> possibly related to memory, as the crash happens more frequently when the
>>> memory size of the VM is relatively small. Thanks.
>>>
>>>
>>> clang-3.6: error: unable to execute command: Killed
>>>
>>> clang-3.6: error: clang frontend command failed due to signal (use -v to
>>> see
>>> invocation)
>>>
>>> clang version 3.6.2 (tags/RELEASE_362/final 252038)
>>>
>>> Target: x86_64-unknown-linux-gnu
>>>
>>> Thread model: posix
>>>
>>> clang-3.6: note: diagnostic msg: PLEASE submit a bug report to
>>> http://llvm.org/bugs/ and include the crash backtrace, preprocessed
>>> source,
>>> and associated run script.
>>>
>>> ********************
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



-- 
GCS a+ e++ d- C++ ULS$ L+$ !E- W++ P+++$ w++$ tv+ b++ DI D++ 5++
The Tao of math: The numbers you can count are not the real numbers.
Life is complex, with real and imaginary parts.
"Ok, it boots. Which means it must be bug-free and perfect. " -- Linus Torvalds
"People disagree with me. I just ignore them." -- Linus Torvalds