[llvm-dev] OrcV1 removal

[Andres Freund via llvm-dev]

Hi,

On 2020-09-25 16:38:41 -0700, Andres Freund via llvm-dev wrote:
> On 2020-09-24 16:34:30 -0700, Lang Hames wrote:
> > If anyone wants to check out the OrcV1 removal branch and provide feedback
> > now is the time. Otherwise I will aim to land the work in the mainline
> > early next week.
> 
> I'm trying to get it to work with postgres. Unfortunately this week was a bit less
> productive than I had hoped... Hope to get there Mon/Tues.

Have the basics working. Should I open a PR to your branch for what I
got so far?


There's currently two things I am still missing from the C API.

The first is to expose more of symbol resolution. Postgres can be
extended via dynamically loaded libraries, and we need to be careful to
look up symbols in the correct library. In the generated JIT IR
references to functions in such libraries use a distinguishable symbol
name (see example below), indicating that they should not be looked up
in the global namespace.

Previously we used the symbol resolution callback to
LLVMOrcAddEagerlyCompiledIR to implement this. If a symbol name was
scoped to a library, we only searched inside that, rather than globally:

/*
 * Attempt to resolve symbol, so LLVM can emit a reference to it.
 */
static uint64_t
llvm_resolve_symbol(const char *symname, void *ctx)
{
...
	llvm_split_symbol_name(symname, &modname, &funcname);
...

	if (modname)
		addr = (uintptr_t) load_external_function(modname, funcname,
							 true, NULL);
	else
		addr = (uintptr_t) LLVMSearchForAddressOfSymbol(symname);
...
	return (uint64_t) addr;
}

I don't think that can be implemented with the current C API.


Related to that, when hitting this case I currently get, on stderr:
JIT session error: Symbols not found: [ pgextern./home/andres/build/postgres/dev-assert/vpath/src/test/regress/regress.so.pt_in_widget ]

but currently that error is not bubbled up to the C API. There I just
get:
ERROR:  XX000: failed to look up symbol "evalexpr_0_0": Failed to materialize symbols: { (main, { evalexpr_0_0 }) }

(obviously the first part is postgres code)

Do you have thoughts on what that should look like?


The other part that I am still missing is replacment for

LLVMCreateGDBRegistrationListener();
LLVMCreatePerfJITEventListener();
LLVMOrcRegisterJITEventListener(llvm_opt0_orc, l);

I haven't yet looked at whether there is replacement infrastructure for
this already. Is there?  Will look after the next few meetings...

Greetings,

Andres Freund