[LLVMdev] Testing methods

[Richard Barton]

Hi James

We are currently finding and fixing bugs in the ARM backends by exhaustively checking all instructions against a trusted reference implementation.

We will start with a 32-bit bitpattern and transform it through a combination of phases in MC and our reference implementation to another bitpattern, which we can compare with the original one.

For example, take a bitpattern, decode it into an MCInst, then disassemble that to an assembly string. You can then transform this string back into a bitpattern using a trusted assembler implementation.

For well defined instructions, if the two do not compare identical then there is a bug somewhere in MC.

We use the proprietary ARM Compiler 5 tool as our reference implementation, so we cannot share the testing infrastructure itself. It should be possible to build a similar approach around another tool, e.g. GNU as.

I gave a talk on our approach at last year's Euro LLVM meeting in London, which you can see on the LLVM site: http://llvm.org/devmtg/2012-04-12/

Regards,
Rich


-----Original Message-----
From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of James Courtier-Dutton
Sent: 21 April 2013 11:22
To: llvmdev at cs.uiuc.edu
Subject: [LLVMdev] Testing methods

Hi,

What does llvm use for testing.
The area of testing I am interested in are how to test the accuracy of the assembler/disassembler.
So, if you take an IR level instruction. How do you verify that the generated CPU specific instruction is correct?
Is there an automated method for this?
I wish to implement automated testing on a decompiler I am writing, and thought that the test methods used in LLVM might be applicable to my decompiler. ( https://github.com/jcdutton/libbeauty ) The thoughts I have had is to use a 3rd party assembler/disassember such as libbfd and parse the results of the 3rd party one, and compare it to mine.

Any help with ideas in this area would help me.

Kind Regards

James
_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.