[LLVMdev] CodeGen runtime tests

[David Blaikie]

On Mon, Apr 20, 2015 at 8:43 AM, Paweł Bylica <chfast at gmail.com> wrote:
> On Mon, Apr 20, 2015 at 5:17 PM David Blaikie <dblaikie at gmail.com> wrote:
>>
>> On Mon, Apr 20, 2015 at 6:07 AM, Paweł Bylica <chfast at gmail.com> wrote:
>> > Hi LLVMdev,
>> >
>> > I have a test that checks if CodeGen implements shift instruction for
>> > illegal types correctly. The test itself checks the runtime result of
>> > the
>> > operation using lli tool. This runtime check approach is quite different
>> > from tests in test/CodeGen, as they use llc tool mainly.
>>
>> Why are you testing it that way, rather than the way other
>> test/CodeGen tests work - verifying assembly?
>
>
> Before the bug was fixed I haven't known what assembly to expect. I can now
> write a test that checks the assembly but the test will be both target and
> implementation specific. Do you think it is better?

generally, yes (yeah, it's not perfect & everything has tradeoffs -
but that's generally the kind of testing that's preferred)

- David

>
>>
>>
>> The regression suite generally (with some exceptions, like the JIT)
>> doesn't execute newly-compiled code. We just test that we produce the
>> right code, not that it executes correctly. (we do this for speed,
>> isolation/stability, etc) We do have the "nightly test suite" (in the
>> test-suite repository) that handles executing arbitrary code examples,
>> etc - but that's mostly for large-ish programs with standalone
>> functionality, not targeted tests.
>>
>> >
>> > Should my test be places somewhere else? Or should it be reimplemented?
>> > http://reviews.llvm.org/D7752
>> >
>> > - Paweł
>> >
>> > _______________________________________________
>> > LLVM Developers mailing list
>> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> >