[LLVMdev] DWARF 2/3 backwards compatibility?

[Rick Foos]

On 10/18/2012 10:36 AM, David Blaikie wrote:
> On Thu, Oct 18, 2012 at 12:48 AM, Renato Golin<rengolin at systemcall.org>  wrote:
>> On 18 October 2012 05:11, Rick Foos<rfoos at codeaurora.org>  wrote:
>>> I don't think GDB testsuite should block a commit, it can vary by a few
>>> tests, they rarely if ever all pass 100%. Tracking the results over time can
>>> catch big regressions, as well as the ones that slowly increase the failed
>>> tests.
>> Agreed. It should at least serve as comparison between two branches,
>> but hopefully, being actively monitored.
>>
>> Maybe would be good to add a directory (if there isn't one yet) to the
>> testsuite repository, or at least the code necessary to make it run on
>> LLVM.
>
> The clang-tests repository (
> http://llvm.org/viewvc/llvm-project/clang-tests/ ) includes an Apple
> GCC 4.2 compatible version of the GCC and GDB test suites that Apple
> run internally. I'm working on bringing up an equivalent public
> buildbot at least for the GDB suite here (
> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-gdb-gcc ) -
> just a few timing out tests I need to look at to get that green.
> Apparently it's fairly stable.
>
> Beyond that I'll be trying to bring up one with the latest suite (7.4
> is what I've been playing with) on Linux as well.
>
Since you're going to bring a bot up in zorg, I'll stop working on 
bringing mine testsuite runner forward. A couple thoughts:

1) I've been running on the latest test suite, polling once a day. I 
think Eric and anyone working dwarf 4/5 should be running against the 
upstream testsuite. (I have no problems with running 7.4 too)

It's been stable to run at the tip of GDB this way, the test results 
aren't varying much.

2) A surprise benefit of running this way is that hundreds of obsolete 
tests, or broken tests are getting removed. This hasn't resulted in any 
broken backwards compatibility here at least. Saves tons of time 
debugging tests that don't work, and developing around compatible things 
that reasonable people have decided no longer matter.

3) Testsuite runs against two compilers at a time makes it easier to see 
regressions. By comparing against a known stable compiler, or GCC, 
regressions are visible by summary numbers.

4) I have plots of the summary numbers online with a window of a month 
or two. The trend allows you to see regressions occurring, and remaining 
as regressions. Sometimes GDB Testsuite or a compiler has a bad day. The 
trend let's you see a stable regression, and when you get a round toit, 
tells you when the regression started.

<soapbox>
I've been doing this with Jenkins. It's fairly easy to set up, and does 
the plotting. Developers can grab a copy of the script to duplicate a 
run on their broken compiler. Running the testsuite under JNLP increased 
the number of executed tests - don't know why just did.
</soapbox>


> - David
>
>> --
>> cheers,
>> --renato
>>
>> http://systemcall.org/
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev


-- 
Rick Foos
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation