<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Jean-Michaël,<div><br></div><div>Sorry -- I misread your email earlier:</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> When ran from *within the container* it works.</blockquote><div><br></div><div>Ahh -- I should be looking for success here. I see why the failure is happening: The testcase doesn't check errors or expected values. You can't ignore those: They have embedded calls to abort in their destructor if you ignore them. For the purpose of writing minimal tests you can always wrap your calls in 'cantFail'. E.g.:</div><div><br></div><div><font face="monospace">auto JIT_e = cantFail(llvm::orc::LLJITBuilder().create());</font><br></div><div><br></div><div>That will strip Expected<T> / Error return types (to T/void), asserting that the value is success in each case.</div><div><br></div><div>Once I make those changes I'm seeing the test pass in my Arch Linux container. Could you share the bitcode that is failing for you? That will help me pin down where things are going off the rails (or failing to go off the rails) with my setup.</div><div><br></div><div>-- Lang.</div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 9, 2020 at 3:11 PM Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Jean-Michaël,<div><br></div><div>Sorry for the delayed reply -- The dev meeting kept me pretty busy the last couple of days.</div><div><br></div><div>When I run your repro.sh script (thank you for taking the time to add all the container config code by the way, that's very helpful!) I see:</div><div><br></div><div><div><font face="monospace">% ./repro.sh </font></div><div><font face="monospace"> -- Starting docker container -- </font></div><div><font face="monospace"><br></font></div><div><font face="monospace">Using default tag: latest</font></div><div><font face="monospace">latest: Pulling from ossia/score-package-linux</font></div><div><font face="monospace">Digest: sha256:aca4c255d4d5a6926e5cd4f50a1a57f6e262a3c931efaaf94a62066784e5424c</font></div><div><font face="monospace">Status: Image is up to date for ossia/score-package-linux:latest</font></div><div><font face="monospace"><a href="http://docker.io/ossia/score-package-linux:latest" target="_blank">docker.io/ossia/score-package-linux:latest</a></font></div><div><font face="monospace"> -- Compiling example to bc --</font></div><div><font face="monospace"><br></font></div><div><font face="monospace"> -- Building -- </font></div><div><font face="monospace"><br></font></div><div><font face="monospace">-- The CXX compiler identification is Clang 11.0.0</font></div><div><font face="monospace">-- Detecting CXX compiler ABI info</font></div><div><font face="monospace">-- Detecting CXX compiler ABI info - done</font></div><div><font face="monospace">-- Check for working CXX compiler: /opt/score-sdk/llvm/bin/clang++ - skipped</font></div><div><font face="monospace">-- Detecting CXX compile features</font></div><div><font face="monospace">-- Detecting CXX compile features - done</font></div><div><font face="monospace">-- Found LLVM 11.0.0</font></div><div><font face="monospace">-- Using LLVMConfig.cmake in: /opt/score-sdk/llvm/lib/cmake/llvm</font></div><div><font face="monospace">-- Configuring done</font></div><div><font face="monospace">-- Generating done</font></div><div><font face="monospace">-- Build files have been written to: /repro/build</font></div><div><font face="monospace">Scanning dependencies of target repro_ffastmath_llvm</font></div><div><font face="monospace">[ 50%] Building CXX object CMakeFiles/repro_ffastmath_llvm.dir/main.cpp.o</font></div><div><font face="monospace">[100%] Linking CXX executable repro_ffastmath_llvm</font></div><div><font face="monospace">[100%] Built target repro_ffastmath_llvm</font></div><div><font face="monospace"> -- Running from within the container : ok</font></div><div><font face="monospace"><br></font></div><div><font face="monospace">/repro/build.sh: line 48: 45 Illegal instruction ./repro_ffastmath_llvm</font></div><div><font face="monospace"> -- Leaving container --</font></div><div><font face="monospace"><br></font></div><div><font face="monospace"> -- Running from within the host system fails: </font></div><div><font face="monospace"><br></font></div><div><font face="monospace">./repro.sh: line 10: ./repro_ffastmath_llvm: cannot execute binary file</font></div></div><div><br></div><div>---</div><div><br></div><div>Is that the same failure that you're seeing for this reproduction case? I was expecting "Symbols not found: [ __log_finite, __exp2_finite ]".</div><div><br></div><div>-- Lang.</div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 5, 2020 at 9:23 PM Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Jean-Michaël,<div><br></div><div>Thanks very much for the reproduction case. I'll try this out tomorrow.</div><div><br></div><div>If you can't take the address of __<F>_finite, then what about defining your own <F>_wrapper functions and using their addresses when defining the absolute symbols? I'm not sure what the performance implications would be for your use-case though. </div><div><br></div><div>-- Lang.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 5, 2020 at 3:23 PM Jean-Michaël Celerier <<a href="mailto:jeanmichael.celerier@gmail.com" target="_blank">jeanmichael.celerier@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hello,</div><div>here is a repro which runs in a docker image.</div><div><a href="https://we.tl/t-O1EhIAOeOF" target="_blank">https://we.tl/t-O1EhIAOeOF</a></div><div><br></div><div>To see the issue, run repro.sh</div><div>It will first download a (big, sorry) centos:7 docker image with my build of LLVM-11 and build a simple lljit-based example.</div><div><br></div><div>This example is called with some trivial .cpp which calls `cos`.</div><div>When ran from *within the container* it works.</div><div>When the same example, with the same bitcode input, runs from outside the container, it does not find this symbol, <br></div><div>likely because the host (in my case Arch, I think you need a glibc-2.31 at least for that behaviour to be visible)'s glibc symbols <br></div><div>became versioned. <br></div><div>Removing either the -fmath-errno or -ffinite-math-only flag for the clang cpp -> bitcode invocation in build.sh fixes the issue</div><div>(at the expense of potentially slower code).<br></div><div><br><div><div dir="ltr"><div dir="ltr"><div><font size="2" face="arial, helvetica, sans-serif" color="#134f5c"><a href="http://www.jcelerier.name" target="_blank"></a></font></div></div></div></div></div><div>Thanks for the hint, sadly it's not possible to take the address of __log_finite : what happens is that you call the function e.g. log() <br></div><div>in your code, and either clang or some magic glibc header transforms that into __log_finite further down the pipeline <br></div><div>(see e.g. the discussion in <a href="https://reviews.llvm.org/D74712" target="_blank">https://reviews.llvm.org/D74712</a> - sadly in my case I can't "upgrade" the headers used by my JIT SDK to glibc-2.31+</div><div> as it would mean that only people with very very recent distros would be able to run the code that's being jit-compiled.</div><div><br></div><div>Thanks !</div><div><br></div><div>Jean-Michaël<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 5, 2020 at 10:11 PM Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Jean-Michaël,<div><br></div><div>Ok -- if you're linking against other symbols without issue then your setup sounds good.</div><div><br></div><div>My first take is that if you're set up correctly then this should "just work", and this failure should be considered a bug, but I need to understand more about ELF indirect / versioned symbols before I can say that definitively. I usually develop on MacOS, but I'll set up a VM and see if I can reproduce this locally to get some more insight here.</div><div><br></div><div>In the meantime one workaround would be to define absoluteSymbol entries for these functions:</div><div><br></div><div><font face="monospace">auto Err = J->getMainJITDylib().define(</font></div><div><font face="monospace"> absoluteSymbols({</font></div><div><div style="color:rgb(0,0,0)"><font face="monospace"> { J->mangleAndIntern("__log_finite"), pointerToJITTargetAddress(&__log_finite) },</font></div><div style="color:rgb(0,0,0)"><font face="monospace"> { J->mangleAndIntern("__exp2_finite"), pointerToJITTargetAddress(&__exp2_finite) }</font></div></div><div style="color:rgb(0,0,0)"><font face="monospace"> }));</font></div><div><br></div><div>-- Lang.</div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 5, 2020 at 12:31 PM Jean-Michaël Celerier <<a href="mailto:jeanmichael.celerier@gmail.com" target="_blank">jeanmichael.celerier@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hello, <br></div><div>Right now I am just using a Generator to look for symbols in my process (which links dynamically against libc / libm). <br></div><div>It seems to have no trouble finding every other libc / libm / libc++ / ... symbol so I assumed that it was not necessary to specifically link against libm where these __finite symbols reside: <br></div><div><br></div><div> $ nm -D /usr/lib/libm.so.6 | grep finite <br> 0000000000050540 T __acosf128_finite@GLIBC_2.26<br> 0000000000042f70 T __acosf_finite@GLIBC_2.15<br> 0000000000026940 i __acos_finite@GLIBC_2.15<br> 0000000000051000 T __acoshf128_finite@GLIBC_2.26<br> 0000000000043240 T __acoshf_finite@GLIBC_2.15<br></div><div><div><div dir="ltr"><div dir="ltr"><div>)</div><div>but maybe it needs some help on that regard ? <br></div><div><br></div><div>Thanks for your quick answer,</div><div><br></div><div>Jean-Michaël<br></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 5, 2020 at 7:53 PM Lang Hames <<a href="mailto:lhames@gmail.com" target="_blank">lhames@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi Jean-Michaël,<div><br></div><div>How are you trying to provide those symbols to the JIT? Are you using a DynamicLibrarySearchGenerator to reflect process symbols (or this specific library's symbols) into the JIT?</div><div><br></div><div>I haven't looked at ELF symbol indirection before -- I'll need to read up on that before I can provide a sensible answer. It's quite likely that RuntimeDyld doesn't support it yet though. Depending on what is required we can either try to implement it there, or aim to fix it in the newer JITLink linker -- a few people are working on an initial implementation of that at the moment.</div><div><br></div><div>-- Lang.</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 5, 2020 at 12:52 AM Jean-Michaël Celerier via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hello,</div><div>when building code with -Ofast -ffinite-math-only -ffast-math, clang generates calls to "finite" variants of math functions.</div><div><br></div><div>This has been the source of a fair amount of issues in a "normal", non-JIT pipeline, which seem to have been fixed over time - a simple fix being recompiling the target app against the new glibc. <br><div>- <a href="https://bugs.llvm.org/show_bug.cgi?id=44842" target="_blank">https://bugs.llvm.org/show_bug.cgi?id=44842</a> <br></div><div>- <a href="https://github.com/cms-sw/cmssw/issues/24935" target="_blank">https://github.com/cms-sw/cmssw/issues/24935</a></div></div><div>- <a href="https://github.com/google/filament/issues/2146" target="_blank">https://github.com/google/filament/issues/2146</a></div><div><br></div><div>But when going through LLJIT (tested with LLVM-10 & LLVM-11, on ArchLinux, glibc-2.32) I still get <br></div><div><br></div><div> Symbols not found: [ __log_finite, __exp2_finite ]</div><div><br></div><div>when trying to materialize my code.</div><div><br></div><div>What could be done for that ? "Recompiling" doesn't seem to fix anything in this case so it looks like LLJIT lacks the mechanism to understand the ELF symbol indirection.<br></div><div><br></div><div>Thanks,</div><div>Jean-Michaël<br> </div></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>