<div dir="ltr">Thanks Nirav,<div><br></div><div>I can confirm that this works when I do the compile with llc, but then when linking to an executable with clang (patched with <a href="http://reviews.llvm.org/D20352">http://reviews.llvm.org/D20352</a> and compiler-rt patched with <a href="http://reviews.llvm.org/D21612">http://reviews.llvm.org/D21612</a>) on Linux, I'm getting something different. Here's a sample of the transcript, and what I'm seeing:</div><div><br></div><div>--->8 clang invocation 8<---</div><div><div>[16-06-23 3:33:42] dberris@dberris: ~/xray/llvm-build% ./bin/clang -fxray-instrument -x c++ -std=c++11 -o test.bin test.cc -g --verbose</div><div>clang version 3.9.0 (<a href="http://llvm.org/git/clang.git">http://llvm.org/git/clang.git</a> 3ae26ac8b1c9c5db65f3dc0236139448b8b0520a) (<a href="http://llvm.org/git/llvm.git">http://llvm.org/git/llvm.git</a> 8fd5dd6aa8a633eeb03b245cd0060479371fc521)</div><div>Target: x86_64-unknown-linux-gnu</div><div>Thread model: posix</div><div>InstalledDir: /usr/local/google/home/dberris/xray/llvm-build/./bin</div><div>Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.7</div><div>Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.7.3</div><div>Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8</div><div>Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.4</div><div>Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9</div><div>Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.3</div><div>Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8</div><div>Candidate multilib: .;@m64</div><div>Candidate multilib: 32;@m32</div><div>Candidate multilib: x32;@mx32</div><div>Selected multilib: .;@m64</div><div> "/usr/local/google/home/dberris/xray/llvm-build/bin/clang-3.9" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -main-file-name test.cc -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -v -dwarf-column-info -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -fxray-instrument -resource-dir /usr/local/google/home/dberris/xray/llvm-build/bin/../lib/clang/3.9.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/x86_64-linux-gnu/c++/4.8 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/x86_64-linux-gnu/c++/4.8 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/backward -internal-isystem /usr/local/include -internal-isystem /usr/local/google/home/dberris/xray/llvm-build/bin/../lib/clang/3.9.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -std=c++11 -fdeprecated-macro -fdebug-compilation-dir /usr/local/google/home/dberris/xray/llvm-build -ferror-limit 19 -fmessage-length 272 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/test-03d46e.o -x c++ test.cc</div><div>clang -cc1 version 3.9.0 based upon LLVM 3.9.0svn default target x86_64-unknown-linux-gnu</div><div>ignoring nonexistent directory "/include"</div><div>ignoring duplicate directory "/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/x86_64-linux-gnu/c++/4.8"</div><div>#include "..." search starts here:</div><div>#include <...> search starts here:</div><div> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8</div><div> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/x86_64-linux-gnu/c++/4.8</div><div> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/backward</div><div> /usr/local/include</div><div> /usr/local/google/home/dberris/xray/llvm-build/bin/../lib/clang/3.9.0/include</div><div> /usr/include/x86_64-linux-gnu</div><div> /usr/include</div><div>End of search list.</div><div> "/usr/bin/ld" -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o test.bin /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.8/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/4.8 -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../.. -L/usr/local/google/home/dberris/xray/llvm-build/bin/../lib -L/lib -L/usr/lib -whole-archive /usr/local/google/home/dberris/xray/llvm-build/bin/../lib/clang/3.9.0/lib/linux/libclang_rt.xray-x86_64.a -no-whole-archive /tmp/test-03d46e.o --no-as-needed -lpthread -lrt -lm -latomic -ldl -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/4.8/crtend.o /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crtn.o</div></div><div>--->8 clang invocation 8<---<br></div><div><br></div><div>The test.cc is simply:</div><div><br></div><div>--->8 test.cc 8<---</div><div>#include <cstdio></div><div>#include <cassert></div><div><br></div><div>[[clang::xray_always_instrument]] void foo() { std::printf("Hello, XRay!\n"); }</div><div><br></div><div>void bar() { std::printf("Not instrumented\n"); }</div><div><br></div><div>extern "C" {</div><div>extern int __xray_patch();</div><div>}</div><div><br></div><div>int main(int argc, char* argv[]) {</div><div> printf("main has started.\n");</div><div> bar();</div><div> foo();</div><div> __xray_patch();</div><div> foo();</div><div>}</div><div>--->8 test.cc 8<---</div><div><br></div><div>A snippet of the disassembly (llvm-objdump -disassemble test.bin) looks like:</div><div><br></div><div>--->8 disassembly 8<---</div><div><div>_Z3foov:</div><div> 400cb0: e9 09 00 00 00 jmp 9 <_Z3foov+0xE></div><div> 400cb5: 66 0f 1f 84 00 00 02 00 00 nopw 512(%rax,%rax)</div><div> 400cbe: 55 pushq %rbp</div><div> 400cbf: 48 89 e5 movq %rsp, %rbp</div><div> 400cc2: 48 83 ec 10 subq $16, %rsp</div><div> 400cc6: 48 bf c5 0e 40 00 00 00 00 00 movabsq $4198085, %rdi</div><div> 400cd0: b0 00 movb $0, %al</div><div> 400cd2: e8 a9 f9 ff ff callq -1623 <.plt+0x30></div><div> 400cd7: 89 45 fc movl %eax, -4(%rbp)</div><div> 400cda: 48 83 c4 10 addq $16, %rsp</div><div> 400cde: 5d popq %rbp</div><div> 400cdf: c3 retq</div><div> 400ce0: 2e 66 0f 1f 84 00 00 02 00 00 nopw %cs:512(%rax,%rax)</div><div> 400cea: 66 0f 1f 44 00 00 nopw (%rax,%rax)</div></div><div><br></div><div>--->8 disassembly 8<---</div><div><br></div><div>Having looked at this a bit, I think you're right that the jumps are being relaxed, due to the -mrelax-all option being used by clang. The question becomes whether it's possible to inhibit relaxation for specific instructions at the LLVM level.</div><div><br></div><div>Cheers</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Jun 22, 2016 at 9:37 AM Nirav Davé <<a href="mailto:niravd@google.com">niravd@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hmm. Odd. I just rebuilt from scratch and it seems to work with the test/CodeGen/X86/xray-attribute-instrumentation.ll test case outputing straight to obj:</div><div><br></div><div> llc -filetype=obj -o ~/a.o -mtriple=x86_64-apple-macosx < test/CodeGen/X86/xray-attribute-instrumentation.ll</div><div><br></div><div>What test case are you using? </div><div><br></div><div>In any case, the issue appears to be that llvm doesn't realize that the target address is resolved and erroneously applies branch relaxation to the jump. I don't know why a linker private symbol would make a difference. </div></div><div dir="ltr"><div><br></div><div>-Nirav</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 22, 2016 at 12:14 PM, Dean Michael Berris <span dir="ltr"><<a href="mailto:dberris@google.com" target="_blank">dberris@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><span><div dir="ltr">On Wed, Jun 22, 2016 at 6:05 AM Nirav Davé <<a href="mailto:niravd@google.com" target="_blank">niravd@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>This appears to work: </div><div><br></div>auto Target = OutContext.createLinkerPrivateTempSymbol();<div><br></div><div>with </div><div><br></div><div><div>auto Target = OutContext.createTempSymbol();</div></div><div><br></div><div>-Nirav<br></div><div><br></div></div></blockquote><div><br></div></span><div>Thanks Nirav -- I tried this but I'm still getting a "jmpq <address>" with this incantation when I load and disassemble from gdb. I'm seeing a 5-instruction jump, followed by the nops.</div><div><br></div><div>If I disassemble with llvm-objdump though I see the following:</div><div><br></div><div><div>_Z3foov:</div><div> 400c10: e9 09 00 00 00 jmp 9 <_Z3foov+0xE></div><div> 400c15: 66 0f 1f 84 00 00 02 00 00 nopw 512(%rax,%rax)</div></div><div><br></div><div>I'm not sure whether the extra 0's after '0xe9 0x09' are alignment padding (though I was expecing 0x90 to show up if this was an alignment issue).</div><div><br></div><div>Is there anything else I can try here?</div><div><br></div><div>Thanks in advance!</div></div></div>
</blockquote></div><br></div>
</blockquote></div>