[cfe-dev] Using clang internals to preprocess a buffer, crashing with double free

[Jonathan Roelofs via cfe-dev]

On 2/23/17 5:24 PM, Larry Gritz via cfe-dev wrote:
> Hi, helpful people. I'm trying to use clang internals to run an in-memory buffer through the C preprocessor (string to string), and I'm having trouble. I wonder if one of you kind souls might be able to spot the problem.
>
> I cobbled this code together looking at clang's BufferSourceTest.cpp unit test (for the aspect of how to read from a buffer), and for how to invoke just the preprocessor, I looked at the source code for ISPC (https://github.com/ispc/ispc/blob/master/module.cpp#L2453).
>
> The C++11 code below, linked against clang/llvm 3.9, works just fine if I compile my code with clang (any of 3.5, 3.9, 4.0) or with g++ 4.8.5. But when I build my code with g++ 4.9, 5.4, or 6.2, it seems to compile fine but will crash at runtime during the exit of this function, with the following error:
>
> *** glibc detected *** myapp: double free or corruption (!prev): 0x0000000000a13270 ***
> ======= Backtrace: =========
> /lib/x86_64-linux-gnu/libc.so.6(+0x7da26)[0x2ac395bf8a26]
> mylib.so(_ZN5clang22CompilerInvocationBaseD1Ev+0xfa)[0x2ac3936ecdca]
> mylib.so(+0x3718e1)[0x2ac3936e88e1]
> mylib.so(_ZN5clang16CompilerInstanceD1Ev+0x445)[0x2ac3936d8f65]
>
> This is on Linux. I don't get these errors on OSX, but then again, I am compiling my code with clang there, and it also doesn't seem to crash when I build my code with clang on Linux either. Only with g++, and only when >= 4.9.
>
> The fact that it crashes for some gcc versions but not others makes me suspicious that it's some misinterpretation of C++11 RAAI, like I'm freeing something that is assumed to be owned by the clang::CompilerInstance. But I'm not sure what that would be.
>
> Anyway, here's the code. I've simplified a bit by removing the parts that deal with -D and -I directives.
> Have I done something foolish here?
>
>
> bool preprocess_buffer (const std::string &buffer,   // Input to be preprocessed
>                         const std::string &filename, // Name for reporting errors
>                         std::string &result)         // <-- want preprocessed output here
> {
>     std::unique_ptr<llvm::MemoryBuffer> mbuf (llvm::MemoryBuffer::getMemBuffer(buffer, filename));
>
>     clang::CompilerInstance inst;
>     inst.createDiagnostics();
>
>     const std::shared_ptr<clang::TargetOptions> &targetopts =
>           std::make_shared<clang::TargetOptions>(inst.getTargetOpts());

This line ^ is extremely suspicious. It makes a shared pointer, takes a 
*reference* to it, and drops it on the floor. To fix it, get rid of the 
'const' and '&' from that line.


Jon

>     targetopts->Triple = llvm::sys::getDefaultTargetTriple();
>     clang::TargetInfo *target =
>         clang::TargetInfo::CreateTargetInfo(inst.getDiagnostics(), targetopts);
>     inst.setTarget(target);
>
>     inst.createFileManager();
>     inst.createSourceManager(inst.getFileManager());
>     clang::SourceManager &sm = inst.getSourceManager();
>     sm.setMainFileID (sm.createFileID(std::move(mbuf), clang::SrcMgr::C_User));
>
>     inst.getPreprocessorOutputOpts().ShowCPP = 1;
>     inst.getPreprocessorOutputOpts().ShowMacros = 0;
>     inst.getLangOpts().LineComment = 1;
>
>     // Omit handling of -D and -I arguments here... not pertinent to the crash
>
>     inst.createPreprocessor(clang::TU_Prefix);
>
>     llvm::raw_string_ostream ostream(result);
>     clang::DoPrintPreprocessedInput (inst.getPreprocessor(),
>                                      &ostream, inst.getPreprocessorOutputOpts());
>     return true;
> }
>
>
>
> --
> Larry Gritz
> lg at larrygritz.com
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>

-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded