[LLVMdev] Making LLVM safer in out-of-memory situations

[Becker, Philipp]

Hi Philip,

Thanks for the positive response from all of you!
 
> One question: How are you handling EOM?  Error return?  Custom region allocator?

When running into an Out-of-memory situation we're currently only doing an error return, i.e. the compilation fails, but does so without crashing the process in which the compilation/jitting occurs. It is ok for us if llvm returns with a catchable exception and unwinds all allocated memory correctly.

To increase stability for us we have already moved the main part of the compilation to a separate process that may crash in case of an error without doing much harm, i.e. does not crash the database. Therefore, we've currently concentrating on specific components that still remain in the database process, such as CodeLoader and VMCore, which is used for emitting IR code. Although, of course, we're also interested in increasing the general stability of the whole llvm w.r.t. error situations.

Best regards,
Philipp

-----Original Message-----
From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Philip Reames
Sent: Freitag, 13. Dezember 2013 02:04
To: Gasiunas, Vaidas; LLVM Dev
Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations

On 12/12/13 4:25 AM, Gasiunas, Vaidas wrote:
> Hello,
>
> Philipp Becker and me, Vaidas Gasiunas, are developers at SAP and part of a team developing a C-like domain-specific language for the SAP HANA in-memory database. We use LLVM as a backend to translate our language to native code, primarily on x86-64 platforms. Our programs are created dynamically, compiled and optimized in a running database. As a result of that we have special requirements with respect to response time and safety. In particular, we have to avoid long compile times and must deal with error situations like out-of-memory without crashing or producing memory leaks in the compiler. The compiler performance is especially important since that we must compile generated functions which tend to be rather long - in the range of thousands of LOC per function.
>
> To address these requirements we have developed a set of patches improving performance and malfunction safety of certain compiler passes and would be interested in contributing them at some point. Before proposing concrete changes, we would like to know what the general interest is with respect to making LLVM safer in out-of-memory situations.
I'm hugely in favor of the general direction.  Happy to help by 
reviewing changes and the like.

This type of work was on my long term todo list; I'm thrilled to see 
someone else doing it now.  :)

One question: How are you handling EOM?  Error return?  Custom region 
allocator?

Philip
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev