<div dir="ltr"><span style="font-family:arial,sans-serif;font-size:13px">Hi Everyone,</span><br style="font-family:arial,sans-serif;font-size:13px"><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
For your consideration: A proposal to improve regression test support for RuntimeDyld.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Short version: We can make RuntimeDyld far more testable by adding a trivial pointer-expression language that allows us to describe how memory should look post-relocation. Jump down to "The Proposal" for details.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Long version:</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
Background:</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">For those unfamiliar with it, RuntimeDyld a component of MCJIT, LLVM's JIT compiler infrastructure. MCJIT produces an object file in memory for each module that is JIT'd. RuntimeDyld's job is to apply all the relocations necessary to make the code in the object file runnable. In other words, RuntimeDyld is acting as both the static and dynamic linker for the JIT.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">The Problem:</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
We can't directly test RuntimeDyld at the moment. We currently infer the correctness of RuntimeDyld indirectly from the success of the MCJIT regression tests - if they pass, we assume RuntimeDyld must have done its job right. That's far from an ideal. The biggest issues with it are:</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">(1) Each platform is testing only its own relocations and no others. I.e. X86 testers are testing X86 relocations only. ARM testers are testing ARM relocations only. If someone running on X86 breaks a relocation for ARM they won't see the error in their regression test run - they'll have to wait until an ARM buildbot breaks before they realize anything is wrong. Fixes for platforms that you don't have access to are difficult to test - all you can do is eyeball disassembled memory and see if everything looks sane. This is not much fun.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">(2) Relocations are produced by CodeGen from IR, rather than described directly. That's a lot of machinery to have between the test-case and the final result. It is difficult to know what relocations each IR regression test is testing (and they're often incidental - we don't have a dedicated relocation test set). This also means that if/when the code generator produces different relocation types the existing tests will keep on passing but will silently stop testing the thing they used to test.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">The Proposal:<br></div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
(1) We provide a mechanism for describing how pieces of relocated memory should look immediately prior to execution, and then inspect the memory rather than executing it. This addresses point (1) above: Tests for any platform can be loaded, linked and verified on any platform. If you're coding on X86 and you break an ARM relocation you'll know about it immediately.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">(2) RuntimeDyld test cases should be written in assembly, rather than IR. This addresses point (2) above - we can cut the code generators out and guarantee that we're testing what we're interested in.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">The way to do this is to introduce a simple pointer expression language. This should be able to express things like: "The immediate for this call points at symbol foo".</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Symbolically, what I have in mind would look something like:</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace"> // some asm ...</font></div><div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace"># assert *(inst1 + 1) = foo<br>
</font></div><div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace">inst1:</font></div><div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace"> callq foo</font></div>
<div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace"> // some asm...</font></div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
Here we add the "inst1" label to give us a address from which we can get at the immediate for the call. The " + 1" expression skips the call opcode (we know the size of the opcode ahead of time, since this is assembly and so target-specific).</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">To verify that constraints expressed in this language hold, we can add an expression evaluator to the llvm-rtdyld utility, which is a command-line interface to RuntimeDyld.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I find these things are easier to discuss in the concrete, so I've attached a basic implementation of this idea. The following discussion is in terms of my patch, but I'm very open to tweaking all this.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">The language I've implemented is:</div><div style="font-family:arial,sans-serif;font-size:13px">
<div><br></div><font face="courier new, monospace">test = expr '=' expr<br><br>expr = '*{' number '}' load_addr_expr<br> | binary_expr<br> | '(' expr ')'<br> | symbol<br>
| number<br><br>load_addr_expr = symbol<br> | '(' symbol '+' number ')'<br> | '(' symbol '-' number ')'<br><br>binary_expr = expr '+' expr<br>
| expr '-' expr<br> | expr '&' expr<br> | expr '|' expr<br> | expr '<<' expr<br> | expr '>>' expr</font><div>
<br></div><div>This expression language supports simple pointer arithmetic, shifting, masking and loading. All values are internally held as 64-bit unsigneds, since RuntimeDlyd is designed to support cross-platform linking, including linking for 64-bit targets from a 32-bit host. I think the only stand-out wart is the *{#size}<addr> syntax for loads. This comes from the fact that immediates aren't always 64-bits, so it's not safe to do a 64-bit load: you could read past the end of allocated memory. The #size field indicates how many bytes to read.</div>
<div><br></div><div>This patch adds a "-verify" option to llvm-rtdyld to attach the expression evaluator to a RuntimeDyld instance after linking. When -verify is passed, llvm-rtdyld does not execute any code. Files containing rules are passed via "-check=<filename>" arguments, and rules are read from any line prefixed with the string "# rtdyld-check: ". The intended workflow is modeled on the FileCheck regression tests. </div>
<div><br></div><div>Here's an example of what a test case for a test for an x86-64 PC-relative MACHO_VANILLA relocation would look like:</div><div><br></div><font face="courier new, monospace">; RUN: clang -triple x86_64-apple-macosx10.9.0 -c -o foo.o %s</font></div>
<div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace">; RUN: llvm-rtdyld -verify -check=foo.s foo.o</font></div><div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace">; RUN: rm foo.o</font></div>
<div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace">;</font></div><div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace">; Test an x86-64 PC-relative MACHO_VANILLA relocation.</font></div>
<div style="font-family:arial,sans-serif;font-size:13px"><font face="courier new, monospace"><br></font></div><div style="font-family:arial,sans-serif;font-size:13px"><div><font face="courier new, monospace"> .text<br>
.globl bar<br> .align 16, 0x90<br>bar:<br> retq</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"> .globl foo<br> .align 16, 0x90<br>
foo:</font></div><div><font face="courier new, monospace"># rtdyld-check: *{4}(inst1 - 4) = (bar - inst1) & 0xffffffff<br> callq bar<br>inst1:<br> retq</font></div><div><br></div><div><div><br></div><div>
With this system, we could write targeted regression tests for every relocation type on every platform, and test them on any system. Failures would immediately identify which target and relocation type broke.</div><div><br>
</div></div></div><div style="font-family:arial,sans-serif;font-size:13px">I think this system would massively improve the testability of the RuntimeDyld layer, which is good news in light of the increased usage MCJIT is getting these days.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Please let me know what you think. Comments and critiques are very welcome, both of the language and the proposed workflow.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Cheers,</div><div style="font-family:arial,sans-serif;font-size:13px">Lang.</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">TL;DR: lhames responds to dblaikie's incessant demand for test cases. ;)</div></div>