[LLVMdev] Testing documentation and terminology

[Matthijs Kooijman]

Hi all,

as you might have seen, I just did some restructuring on the testing
documentation. Since the changes are only documentation and mostly text
movement, I took the liberty of committing without posting a patch first. Was
that appropriate?.

The incentive for this commit was that I've spent the last week trying to get
a grip on the test suite with moderate success. I think this restructuring
will help new users to get started with the test suite faster. Or, more
directly, I wanted to add some documentation on comparing test results, but
couldn't find a decent spot in the testing documentation structure.

There is still one issue I'd like to discuss here. The test suite
documentation is a bit confusing in terminology. There are three import
concepts: All of the testing stuff in llvm, the llvm/test DejaGNU part, and
the test-suite part. 

Currently, the first is called the "LLVM test suite" (which is the subject of
the testing documentation). The second is called the DejaGNU tests, code
fragment tests or llvm/test. The last one is the most confusing, it's called
the whole program tests, test-suite (named after the svn directory it is in,
easily confused with the entire test suite), llvm-test (named after the
directory in which you should place it, easily confused with llvm/test).

I propose a few changes here. First, to use the "whole program tests", you
should checkout the "test-suite" svn module into a "projects/llvm-test"
directory.  This would be way more logical if you could just checkout into a
"projects/test-suite" directory.

Furthermore, I would no longer refer to all of llvm's testing stuff as the
"LLVM test suite", but use the term "LLVM test architecture" or "framework" or
something similar (suggestions?). Since the testing stuff is composed of two
distinct parts, I'm not sure the term "suite" is appropriate here. If people
think that "test suite" is still very much appropriate here, I think that
renaming the "test-suite" svn module would be in order.

Lastly, I would refer to the "whole program tests" as the "Test suite", since
it is a suite of testing programs that can be run as a whole. This will also
make sure that the name properly matches the svn module that the "whole
program tests" are housing in.

After these changes, I think the testing framework should be a lot easier to
describe and document and more accessible to new users.

Any comments/questions/objections?

Gr.

Matthijs
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20080523/b9784129/attachment.sig>