[LLVMdev] LLVM memory usage?

[James Williams]

On 12 February 2010 01:48, Reid Kleckner <rnk at mit.edu> wrote:

> On Thu, Feb 11, 2010 at 6:53 PM, James Williams <junk at giantblob.com>
> wrote:
> > Hi,
> >
> > I'm seeing rather high memory usage from LLVM and I'd like to track down
> > what I'm doing to cause it. My application is a simple web application
> > server that compiles web pages with embedded script to bitcode and
> compiles
> > them with the JIT on demand. I've taken tools/lli.cpp as a starting point
> > and extended it to load additional modules.
> >
> > However, if I load successive pages and watch top, my RSS size increases
> by
> > roughly 40 times the on disk size of the loaded bit code for each loaded
> > module, which I take to mean I have a massive leak. I'm probably just not
> > freeing stuff that I ought to but I don't know what - deleting the
> > MemoryBuffer or ModuleProvider results in a SEGV. I've read the doxygen
> docs
> > but I'm not clear what objects have allocated what memory for what
> purpose,
> > what can be freed once all functions in a module have been compiled or
> how
> > to do this.
> >
> > I'd be grateful if anyone could tell me what I'm doing wrong or point me
> at
> > docs or examples covering this topic? The gist of my code is:
> >
> > Module *load_module(char *bitcode_name) {
> >   MemoryBuffer *buffer = MemoryBuffer::getFile(bitcode_name,
> > &error_message);
> >   ModuleProvider *mp = getBitcodeModuleProvider(buffer,
> getGlobalContext(),
> > &error_message);
> >
> >   if( first_time ) {
> >       InitializeNativeTarget();
> >       builder = new EngineBuilder(mp);
> >       builder->setEngineKind(EngineKind::JIT);
> >       CodeGenOpt::Level opt_level = CodeGenOpt::Default;
> >       builder->setOptLevel(opt_level);
> >       execution_engine = builder->create();
> >       delete(builder);       // lli doesn't do this - is it safe?
>
> lli allocates the builder on the stack, so it is automatically
> destroyed on scope exit.
>
OK. My C++ is very rusty!

>
> >   }
> >
> >   Module *module = mp->materializeModule(&error_message);
> >   for (Module::iterator I = module->begin(), E = module->end(); I != E;
> ++I)
> > {
> >     Function *f = &*I;
> >     if(!f->isDeclaration()) {
> >       execution_engine->getPointerToFunction(f);
> >     }
> >   }
> >   // all functions are compiled so I'd like to dispose of the bitcode any
> > memory used in the compilation - how can I do this?
>
> You can call (I think) deleteBody on each compiled function if you're
> done with it to delete the IR and leave the machine code.  You can't
> delete the functions themselves, because their pointers are used as
> keys in a hashtables.
>
Thanks, I'll give this a try

-- James

>
> Reid
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20100212/48730180/attachment.html>