[llvm-dev] IRMover asserts "mapping to a source type" when repeatedly linking - usage or LLVM bug?

[Andres Freund via llvm-dev]

Hi,

(sorry if the CC's are inappropriate, they seemed relevant based on a
git log of IRMover.cpp)

I'm using LLVM to implement Just-in-Time compilation for PostgreSQL. One
part of that is doing inlining of operators. For that I'm using bitcode
pre-generated using clang.

The current code uses a single LLVMContext & Orc to generate the
code. That largely workes well.  But inlining presents a bit of a
problem.

(end of setup)

I've tried two approaches to inlining:

In the first I'm, lazily, caching the source modules, and CloneModule()
them before handing them to IRMover to inline necessary function into
the target module.  That works well in release builds, but occasionally
triggers an assert in debug mode ("mapping to a source type"). More on
the assert later.

The second approach is to *not* cache modules but re-read them from disk
(or memory, but that's irrelevant here). That works without any sort of
asserts, but "leaks" memory because everytime a module is re-read from
disk it re-creates types (cf BitcodeReader.cpp:parseTypeTableBody()).
Creating a different LLVMContext for every single expression used seems
infeasible, and fixing LLVM to have type lifetimes bound to Modules
seems daunting.

Therefore the first approach seems preferable by far.

What I'm observing is that creating the same Module contents (modulo
some constants) twice and then inlining a number of Modules
(selectively, but that seems unrelated), works the first time, but
crashes the second with the "mapping to a source type" assertion.

I'm not 100% sure yet, but my understanding of what happens is the
following:

1) "main1" module gets created
2) inlinining loads module "a", clones it, links module "a'" into
   "main1". This module contains an opaque struct definition "t" that is now
   referenced by "main1'".
3) inlinining loads module "b", clones it, links module "b'" into
   "main1'". This module has a *non-opaque* definition of "t". Thus "t",
   as referenced by "main1'", gets its type "t" completed (via
   linkDefinedTypeBodies()).
4) "main2" module gets created
5) inlining clones module "a". As types are global to a context, "t" now
   has a body, *and* the body appears to be from the source module
   (actually "b'"). "mapping to a source type" is the result.

As far as I can tell, which doesn't say much, I don't know this code
well, the code actually handles this situation correctly. It's just the
assert that triggers that is problematic.

Am I doing something unsupported here or is this a bug?

As far as I can tell there unfortunately is no tool in LLVM that allows
to conveniently create a reproducer, unfortunately. I can write a
short-ish C++ program if that helps.

Greetings,

Andres Freund