[LLVMdev] 3.3 Testers!

[Sebastian Dreßler]

Hi Tyler,

On 05/09/2013 07:03 AM, Tyler Hardin wrote:
> First, let me ask if there's a page that documents the testing process in
> full. If there is, just give me a link to that and save yourself the time
> of answering this. (I did Google it, all I found was docs on writing tests
> for LLVM.)
> 
> Sorry if this is way off, but I have no idea what to download and unpack to
> start testing. I know it's stuff from here:
> http://llvm.org/pre-releases/3.3/rc1/.
> 
> I've built LLVM and Clang before, for my own use. IIRC, I put clang in
> llvm/tools/ and just ran ./configure and make. So what I need to know is:
> which of those files do I need, and which llvm subdirectories the go in?
> 
> And I do this for both 3.3rc1 and 3.2, then run:
> utils/release/test-release.sh --release 3.2 in the llvm 3.2 directory, and
> utils/release/test-release.sh --release 3.3 --rc 1 in the llvm 3.3rc1
> directory, right?
> 
> For clarification, by llvm 3.x directory, I mean the extracted llvm 3.x
> tarball and all the 3.x sub-tools extracted in their respective
> subdirectories.
> 
> Then what after that?
> 
> Sorry for all the extra clarification. This is the first testing I've ever
> done. I really like LLVM and accompanying tools. I'd like to become more
> involved in the LLVM project, but I don't think I have the necessary skills
> yet.
> 

Disclaimer: This is also my very first testing session, but I think that
I've figured it out ... at least the procedure makes sense to me ;)


The full procedure for getting the nightly test results of *a single RC*
is as follows:

1. Get the sources for LLVM + Clang from SVN (i.e. checkout from the
specific tags, e.g. RELEASE_33/rc1)

2. Run the script from utils/release/test-release.sh with appropriate
arguments, e.g. to build 3.3 RC1:

    test-release.sh -release 3.3 -rc 1

Adapt -build-dir and -j as needed

3. Once 2 is finished, download the llvm-test-suite, install LNT (if not
done already) and run it, don't forget to set --cc and --cxx to the
newly compiled clang/clang++ binaries. When this step is finished,
you'll get a directory containing a log file (report.json)


Now for the important: you have to do this for 3.2final *and* 3.3rc1m so
simply repeat steps 1 to 3. Once you have the result.json files for both
releases, you can compare them with the mentioned
findRegressions-{nightly,simple}.py scripts.


I hope that clarifies some things.

Cheers,
Sebastian

-- 
Mit freundlichen Grüßen / Kind regards

Sebastian Dreßler

Zuse Institute Berlin (ZIB)
Takustraße 7
D-14195 Berlin-Dahlem
Germany

dressler at zib.de
Phone: +49 30 84185-261

http://www.zib.de/