[LLVMdev] LLVM memory usage?

[Reid Kleckner]

On Thu, Feb 11, 2010 at 6:53 PM, James Williams <junk at giantblob.com> wrote:
> Hi,
>
> I'm seeing rather high memory usage from LLVM and I'd like to track down
> what I'm doing to cause it. My application is a simple web application
> server that compiles web pages with embedded script to bitcode and compiles
> them with the JIT on demand. I've taken tools/lli.cpp as a starting point
> and extended it to load additional modules.
>
> However, if I load successive pages and watch top, my RSS size increases by
> roughly 40 times the on disk size of the loaded bit code for each loaded
> module, which I take to mean I have a massive leak. I'm probably just not
> freeing stuff that I ought to but I don't know what - deleting the
> MemoryBuffer or ModuleProvider results in a SEGV. I've read the doxygen docs
> but I'm not clear what objects have allocated what memory for what purpose,
> what can be freed once all functions in a module have been compiled or how
> to do this.
>
> I'd be grateful if anyone could tell me what I'm doing wrong or point me at
> docs or examples covering this topic? The gist of my code is:
>
> Module *load_module(char *bitcode_name) {
>   MemoryBuffer *buffer = MemoryBuffer::getFile(bitcode_name,
> &error_message);
>   ModuleProvider *mp = getBitcodeModuleProvider(buffer, getGlobalContext(),
> &error_message);
>
>   if( first_time ) {
>       InitializeNativeTarget();
>       builder = new EngineBuilder(mp);
>       builder->setEngineKind(EngineKind::JIT);
>       CodeGenOpt::Level opt_level = CodeGenOpt::Default;
>       builder->setOptLevel(opt_level);
>       execution_engine = builder->create();
>       delete(builder);       // lli doesn't do this - is it safe?

lli allocates the builder on the stack, so it is automatically
destroyed on scope exit.

>   }
>
>   Module *module = mp->materializeModule(&error_message);
>   for (Module::iterator I = module->begin(), E = module->end(); I != E; ++I)
> {
>     Function *f = &*I;
>     if(!f->isDeclaration()) {
>       execution_engine->getPointerToFunction(f);
>     }
>   }
>   // all functions are compiled so I'd like to dispose of the bitcode any
> memory used in the compilation - how can I do this?

You can call (I think) deleteBody on each compiled function if you're
done with it to delete the IR and leave the machine code.  You can't
delete the functions themselves, because their pointers are used as
keys in a hashtables.

Reid