[llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup?

[Lang Hames via llvm-dev]

Hi Bjoern,


> ... and "?Sampler@@YAXXZ" is stuck as a pending query in the
>> MaterializingInfos entries.
>
>

Huh. That sounds like a bug: All references to the query should be removed
> from the state machine before it returns its result (in this case an
> error). I'll see if I can reproduce this locally and fix it up, but it
> doesn't affect the discussion here.


I tried to reproduce this locally by defining an absolute symbol "foo",
then issuing a lookup for two symbols "foo" and "bar". The error was
produced as expected, but I couldn't see any queries left attached to the
ExecutionSession. Did you definitely see a failed query still attached? If
so, and you see this again could you log your output and file a bug?


-- Lang.

On Tue, Sep 29, 2020 at 9:55 AM Lang Hames <lhames at gmail.com> wrote:

> Hi Bjoern,
>
> However, another thing of our system is, that each object file was loaded
>> from a different process,
>
>
> Do you mean that the object file is produced by another process and is
> being loaded into your JIT process for execution, or that you want your JIT
> to produce code for several different processes? These are different
> problems with different solutions. I'll wait until I understand your use
> case to answer further.
>
> Writing this… I actually wondered about something else but this feels like
>> a terrible approach… But now I’m curious xD
>> So… if I load an IR-Module, could I use the static LLVM compiler to
>> compile it to an object file and then use the source code of the LLD to
>> load and resolve the symbols the same way/kind we did in the past?
>> That sounds more like the “addObjectFile” function of the LLJIT… And then
>> I guess I have to write a LinkLayer or something? That is where my
>> knowledge ends…
>> Disclaimer: I don’t like that approach but it would be interesting to
>> know (also cause some people here would be happy with it .w.”)
>
>
> You're trying to do all this on Hard Mode. ;)
>
> ORC takes care of all this kind of stuff for you:
>   Don't re-write IR. Leave references as symbolic -- they will be fixed up
> in the JIT linker.
>   You don't need to write your own JIT linker. LLJIT has one built in.
>
> When you add things to the JIT:
>   - If you have a program representation (module, object file, etc.) and
> you want to add it then just go ahead.
>   - If your program representations contain external references then they
> must be resolvable or linking will fail (there's no getting around that, in
> a JIT or in a regular compile), BUT...
>   - You can always add new definitions in response to a query by using a
> definition generator. If your definition generator can find/create a
> definition then great. If it can't then the reference really is unresolved
> and linking really should fail.
>
> The part of your use case that is the most opaque to me is the renaming.
> When you see a reference to "test" in some object, how do you decide that
> it should resolve to the definition of "test" in, for example, Planschi, as
> opposed to some other module? Do you just have a list of modules that you
> check in-order until you find a matching symbol name?
>
> -- Lang.
>
> On Tue, Sep 29, 2020 at 2:30 AM Gaier, Bjoern <Bjoern.Gaier at horiba.com>
> wrote:
>
>> Hey Lang,
>>
>>
>>
>> Thank you for your help and your patience – also for your answers in the
>> “ORC JIT - Can modules independently managed with one LLJIT instance? +
>> problems with ExecutionSession.lookup” mail. Both problems have the same
>> origin so I keep writing about it here, to avoid duplication.
>>
>> My big problem is still handling cross references between modules with
>> “our” name scheme. Since our old loader loads object files, we resolved
>> those references with object files and since they were already compiled, we
>> knew all addresses right away. With the LLJIT as I finally understand, I
>> will only get the addresses when I have resolved every references, which
>> makes the code way safer.
>>
>> However, another thing of our system is, that each object file was loaded
>> from a different process, so sometimes not all symbols for ModuleA were
>> present because ModuleB was not loaded/requested yet. That was okay, so we
>> kept resolving the undefined references of ModuleA until ModuleB was loaded
>> and everything was fine.
>>
>>
>>
>> If I get it right… This would change now to having a single LLJIT
>> representing the entire system. Each process would get it’s own DyLib for
>> there module. However, I would need to check on IR-Level now which symbols
>> would be undefined – correct? Because if I wait until
>> “DefinitionGenerator::tryToGenerate” is called and have to wait for a
>> module that might never be loaded, then I’m stuck there forever.
>>
>> 1.) If I find a symbol that is undefined – and it has our name scheme,
>> then I would jump to for example ModuleB, which is also not jitted yet and
>> would do a “replaceAllUsesWith” on the Symbol of ModuleA to ModuleB –
>> right?
>>
>>                 - Would that mean, when I add ModuleA to DyLibA – is
>> ModuleB then part of DyLibA as well?
>>
>> 1.1.) Alternatively I could rename the symbol
>>
>> 2.) If the ModuleB is already jitted, then I can take the address to  do
>> the “replaceAllUsesWith” right?
>>
>>
>>
>> When I resolved all those references, then I can add the IR Module to my
>> DyLib and compile it. However is it a good idea to use “replaceAllUsesWith”
>> with addresses? Seems like the DefinitionGenerator would be jobless…
>>
>>
>>
>> Writing this… I actually wondered about something else but this feels
>> like a terrible approach… But now I’m curious xD
>>
>> So… if I load an IR-Module, could I use the static LLVM compiler to
>> compile it to an object file and then use the source code of the LLD to
>> load and resolve the symbols the same way/kind we did in the past?
>>
>> That sounds more like the “addObjectFile” function of the LLJIT… And then
>> I guess I have to write a LinkLayer or something? That is where my
>> knowledge ends…
>> Disclaimer: I don’t like that approach but it would be interesting to
>> know (also cause some people here would be happy with it .w.”)
>>
>>
>>
>> Thank you so far!
>>
>>
>>
>> Kind greetings
>>
>> Björn
>>
>>
>>
>>
>>
>> *From:* Lang Hames <lhames at gmail.com>
>> *Sent:* 29 September 2020 01:47
>> *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com>
>> *Cc:* LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
>> *Subject:* Re: [llvm-dev] ORC JIT - different behaviour of
>> ExecutionSession.lookup?
>>
>>
>>
>> Hi Bjoern,
>>
>>
>>
>> Even though the "tryToGenerate" function of my DefinitionGenerator
>> returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I
>> got an address for "?
>>
>>
>>
>> That's because you're issuing the lookup with RequiredState ==
>> SymbolState::Resolved. This means that your query will return as soon as
>> "?Sampler@@YAXXZ" is assigned an address. In the JIT linker(s) addresses
>> are assigned before external references are looked up. So after your lookup
>> returns the linker attempts to find "?_Plansch_test@@3HA", fails, and so
>> moves "?Sampler@@YAXXZ" to the error state.
>>
>>
>>
>> You almost always want to issue your lookups with RequiredState ==
>> SymbolState::Ready. This ensures that the query will not return until /
>> unless the requested symbols (and all their dependencies) are successfully
>> linked into the target process and ready to execute.
>>
>>
>>
>> Question 1.)
>> Is there any way to reset the error state of "?Sampler@@YAXXZ" at this
>> point?
>>
>>
>>
>> No. However, the removable code feature will allow you to remove failed
>> materialization units once it lands in the mainline.
>>
>>
>>
>> - After my first call I used the "define" function of the JITDylib to
>> define ?_Plansch_test@@3HA and then I tried calling the lookup function
>> again and again, however I only got the error: "Failed to materialize
>> symbols" even though "?_Plansch_test@@3HA" was defined now...
>> - Changing the order of the "define" and the "lookup" call works of
>> course, but I'm interested in the case where I don't know the address yet.
>>
>>
>>
>> The JIT doesn't re-try linking. Once a symbol has failed to link it
>> remains in the error state. In theory, once removable code is added you
>> could choose to remove and then re-add "?Sampler@@YAXXZ"
>> after "?_Plansch_test@@3HA" is defined. The real solution though is just
>> to make sure that "?_Plansch_test@@3HA" is defined (either directly or
>> via a generator) before you look up "?Sampler@@YAXXZ".
>>
>>
>>
>> Out of curiosity I repeated the previous scenario - but added
>> "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly.
>> When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error
>> from my DefinitionGenerator...
>>
>>
>>
>> Yes. Because "?_Plansch_test@@3HA" is not defined. You should see a
>> SymbolsNotFound error sent to your error reporter in the first scenario
>> too, followed by a failure-to-materialize error for "?Sampler@@YAXXZ".
>>
>>
>>
>> ... and "?Sampler@@YAXXZ" is stuck as a pending query in the
>> MaterializingInfos entries.
>>
>>
>>
>> Huh. That sounds like a bug: All references to the query should be
>> removed from the state machine before it returns its result (in this case
>> an error). I'll see if I can reproduce this locally and fix it up, but it
>> doesn't affect the discussion here.
>>
>>
>>
>> When I then add a definition for "?_Plansch_test@@3HA" and call "lookup"
>> the second time, it will succeed and give me the addresses. Also I'm able
>> to execute the code now. This is great! However...
>>
>>
>>
>> When a lookup fails we try to restore the ExecutionSession state to what
>> it was prior to the query. This is why the sequence "lookup -> symbols not
>> found -> define -> lookup again" worked.
>>
>>
>>
>> Question 2.)
>> Why did the first call to lookup not return the address of "?Sampler@@YAXXZ"
>> like in the first scenario? I expected it would return an address for it.
>>
>>
>>
>> A lookup must match against all symbols before anything is JIT'd. When it
>> failed to match "?_Plansch_test@@3HA" we immediately bailed out with an
>> error. There was no further attempt to compile "?Sampler@@YAXXZ".
>>
>>
>>
>> Question 3.)
>> Can I somehow combine both behaviours? Getting the address for all the
>> symbols (like in scenario 1) while still being able to provide definitions
>> later (like in scenario 2)?
>>
>>
>>
>> *Sort of.*
>>
>>
>>
>> Definition generators allow you to provide a definition at the last
>> minute (i.e. in response to a query). The best mental model though is: "All
>> definitions that a generator can generate are part of the interface of the
>> dylib". E.g. if you use a DynamicLibrarySearchGenerator to mirror symbols
>> from a dynamic library containing "foo", "bar" and "baz" then you should
>> think of your JITDylib as containing definitions for "foo", "bar" and
>> "baz", even if the generator hasn't actually added them to the JITDylib
>> yet. The reason is that it will add them in response to any query for them,
>> so it's indistinguishable (except for timing and debug logging) from the
>> case where they're already present.
>>
>>
>>
>> If you need to be able to defer adding a "real" definition beyond the
>> initial lookup then your only option (and this only applies to functions)
>> is a lazy-reexport. This allows you to provide a definition for a function
>> while deferring lookup until the first execution of the re-export at
>> runtime. I wouldn't generally use this to break dependencies though: You
>> want a definition of the real function body for "?_Plansch_test@@3HA"
>> already added to your JIT because (in general) you never know when JIT'd
>> code will need it.
>>
>>
>>
>> My turn to ask a question: How is "?_Plansch_test@@3HA" created, and why
>> not just add it up-front? :)
>>
>>
>>
>> -- Lang.
>>
>>
>>
>> On Mon, Sep 28, 2020 at 4:57 AM Gaier, Bjoern via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> Hey everyone,
>>
>>
>>
>> I felt this question is different from my other question - hope this is
>> okay.
>>
>>
>>
>> So - I was playing around with the lookup function of the
>> ExecutionSession and there are some things I don't understand.
>>
>> I have a .BC file with a function "?Sampler@@YAXXZ" referencing a value
>> "?_Plansch_test@@3HA" that is not defined in that module itself. I first
>> planed on not providing an address for "?_Plansch_test@@3HA" but wanted
>> to know the address of "?Sampler@@YAXXZ". So I issued something like
>> that:
>>
>>
>>
>>                 auto &ES = this->jit->getExecutionSession();
>>
>>                 SymbolLookupSet lookupSet;
>>
>>
>>
>>                 lookupSet.add("?Sampler@@YAXXZ",
>> llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol);
>>
>>                 ES.lookup({{&jit->getMainJITDylib(),
>> llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet,
>> llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved);
>>
>>
>>
>> Even though the "tryToGenerate" function of my DefinitionGenerator
>> returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I
>> got an address for "?Sampler@@YAXXZ". Dumping the "MainJITDylib" I saw,
>> that the "?Sampler@@YAXXZ" was in an Error state. Which made sense - I
>> guess.
>>
>>
>>
>> Question 1.)
>>
>> Is there any way to reset the error state of "?Sampler@@YAXXZ" at this
>> point?
>>
>> - After my first call I used the "define" function of the JITDylib to
>> define ?_Plansch_test@@3HA and then I tried calling the lookup function
>> again and again, however I only got the error: "Failed to materialize
>> symbols" even though "?_Plansch_test@@3HA" was defined now...
>>
>> - Changing the order of the "define" and the "lookup" call works of
>> course, but I'm interested in the case where I don't know the address yet.
>>
>>
>>
>> Out of curiosity I repeated the previous scenario - but added
>> "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly.
>> When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error
>> from my DefinitionGenerator and "?Sampler@@YAXXZ" is stuck as a pending
>> query in the MaterializingInfos entries. When I then add a definition for
>> "?_Plansch_test@@3HA" and call "lookup" the second time, it will succeed
>> and give me the addresses. Also I'm able to execute the code now. This is
>> great! However...
>>
>>
>>
>> Question 2.)
>>
>> Why did the first call to lookup not return the address of "?Sampler@@YAXXZ"
>> like in the first scenario? I expected it would return an address for it.
>>
>>
>>
>> Question 3.)
>>
>> Can I somehow combine both behaviours? Getting the address for all the
>> symbols (like in scenario 1) while still being able to provide definitions
>> later (like in scenario 2)?
>>
>>
>>
>> Thank you in advance and kind greetings,
>>
>> Björn
>>
>> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
>> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
>> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
>> Cansiz.
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
>> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
>> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
>> Cansiz.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200929/afbc03c0/attachment-0001.html>