[LLVMdev] Making LLVM safer in out-of-memory situations

[Hal Finkel]

----- Original Message -----
> From: "Philipp Becker" <philipp.becker at sap.com>
> To: "Philip Reames" <listmail at philipreames.com>, "LLVM Dev" <llvmdev at cs.uiuc.edu>
> Sent: Friday, December 13, 2013 6:55:59 AM
> Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations
> 
> Hi Philip,
> 
> Thanks for the positive response from all of you!
>  
> > One question: How are you handling EOM?  Error return?  Custom
> > region allocator?
> 
> When running into an Out-of-memory situation we're currently only
> doing an error return, i.e. the compilation fails, but does so
> without crashing the process in which the compilation/jitting
> occurs. It is ok for us if llvm returns with a catchable exception
> and unwinds all allocated memory correctly.

Does this mean that you're using C++ exception handling to manage the cleanup?

 -Hal

> 
> To increase stability for us we have already moved the main part of
> the compilation to a separate process that may crash in case of an
> error without doing much harm, i.e. does not crash the database.
> Therefore, we've currently concentrating on specific components that
> still remain in the database process, such as CodeLoader and VMCore,
> which is used for emitting IR code. Although, of course, we're also
> interested in increasing the general stability of the whole llvm
> w.r.t. error situations.
> 
> Best regards,
> Philipp
> 
> -----Original Message-----
> From: llvmdev-bounces at cs.uiuc.edu
> [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Philip Reames
> Sent: Freitag, 13. Dezember 2013 02:04
> To: Gasiunas, Vaidas; LLVM Dev
> Subject: Re: [LLVMdev] Making LLVM safer in out-of-memory situations
> 
> On 12/12/13 4:25 AM, Gasiunas, Vaidas wrote:
> > Hello,
> >
> > Philipp Becker and me, Vaidas Gasiunas, are developers at SAP and
> > part of a team developing a C-like domain-specific language for
> > the SAP HANA in-memory database. We use LLVM as a backend to
> > translate our language to native code, primarily on x86-64
> > platforms. Our programs are created dynamically, compiled and
> > optimized in a running database. As a result of that we have
> > special requirements with respect to response time and safety. In
> > particular, we have to avoid long compile times and must deal with
> > error situations like out-of-memory without crashing or producing
> > memory leaks in the compiler. The compiler performance is
> > especially important since that we must compile generated
> > functions which tend to be rather long - in the range of thousands
> > of LOC per function.
> >
> > To address these requirements we have developed a set of patches
> > improving performance and malfunction safety of certain compiler
> > passes and would be interested in contributing them at some point.
> > Before proposing concrete changes, we would like to know what the
> > general interest is with respect to making LLVM safer in
> > out-of-memory situations.
> I'm hugely in favor of the general direction.  Happy to help by
> reviewing changes and the like.
> 
> This type of work was on my long term todo list; I'm thrilled to see
> someone else doing it now.  :)
> 
> One question: How are you handling EOM?  Error return?  Custom region
> allocator?
> 
> Philip
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory