[Lldb-commits] [lldb] r136553 - in /lldb/trunk/test: benchmarks/ benchmarks/example/ benchmarks/example/Makefile benchmarks/example/TestRepeatedExprs.py benchmarks/example/main.c dotest.py lldbtest.py python_api/lldbutil/frame/TestFrameUtils.py python_api/lldbutil/iter/TestLLDBIterator.py python_api/lldbutil/iter/TestRegistersIterator.py python_api/lldbutil/process/TestPrintStackTraces.py

Johnny Chen johnny.chen at apple.com
Fri Jul 29 18:39:58 PDT 2011


Author: johnny
Date: Fri Jul 29 20:39:58 2011
New Revision: 136553

URL: http://llvm.org/viewvc/llvm-project?rev=136553&view=rev
Log:
Add a @benchmarks_test decorator for test method we want to categorize as benchmarks test.
The test driver now takes an option "+b" which enables to run just the benchmarks tests.
By default, tests decorated with the @benchmarks_test decorator do not get run.

Add an example benchmarks test directory which contains nothing for the time being,
just to demonstrate the @benchmarks_test concept.

For example,

$ ./dotest.py -v benchmarks

...

----------------------------------------------------------------------
Collected 2 tests

1: test_with_gdb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with gdb. ... skipped 'benchmarks tests'
2: test_with_lldb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with lldb. ... skipped 'benchmarks tests'

----------------------------------------------------------------------
Ran 2 tests in 0.047s

OK (skipped=2)
$ ./dotest.py -v +b benchmarks

...

----------------------------------------------------------------------
Collected 2 tests

1: test_with_gdb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with gdb. ... running test_with_gdb
benchmarks result for test_with_gdb
ok
2: test_with_lldb (TestRepeatedExprs.RepeatedExprssCase)
   Test repeated expressions with lldb. ... running test_with_lldb
benchmarks result for test_with_lldb
ok

----------------------------------------------------------------------
Ran 2 tests in 0.270s

OK

Also mark some Python API tests which are missing the @python_api_test decorator.

Added:
    lldb/trunk/test/benchmarks/
    lldb/trunk/test/benchmarks/example/
    lldb/trunk/test/benchmarks/example/Makefile
    lldb/trunk/test/benchmarks/example/TestRepeatedExprs.py
    lldb/trunk/test/benchmarks/example/main.c
Modified:
    lldb/trunk/test/dotest.py
    lldb/trunk/test/lldbtest.py
    lldb/trunk/test/python_api/lldbutil/frame/TestFrameUtils.py
    lldb/trunk/test/python_api/lldbutil/iter/TestLLDBIterator.py
    lldb/trunk/test/python_api/lldbutil/iter/TestRegistersIterator.py
    lldb/trunk/test/python_api/lldbutil/process/TestPrintStackTraces.py

Added: lldb/trunk/test/benchmarks/example/Makefile
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/benchmarks/example/Makefile?rev=136553&view=auto
==============================================================================
--- lldb/trunk/test/benchmarks/example/Makefile (added)
+++ lldb/trunk/test/benchmarks/example/Makefile Fri Jul 29 20:39:58 2011
@@ -0,0 +1,5 @@
+LEVEL = ../../make
+
+C_SOURCES := main.c
+
+include $(LEVEL)/Makefile.rules

Added: lldb/trunk/test/benchmarks/example/TestRepeatedExprs.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/benchmarks/example/TestRepeatedExprs.py?rev=136553&view=auto
==============================================================================
--- lldb/trunk/test/benchmarks/example/TestRepeatedExprs.py (added)
+++ lldb/trunk/test/benchmarks/example/TestRepeatedExprs.py Fri Jul 29 20:39:58 2011
@@ -0,0 +1,37 @@
+"""Test evaluating expressions repeatedly comparing lldb against gdb."""
+
+import os
+import unittest2
+import lldb
+import pexpect
+from lldbtest import *
+
+class RepeatedExprssCase(TestBase):
+
+    mydir = os.path.join("benchmarks", "example")
+
+    @benchmarks_test
+    def test_with_lldb(self):
+        """Test repeated expressions with lldb."""
+        self.buildDefault()
+        self.run_lldb_repeated_exprs()
+
+    @benchmarks_test
+    def test_with_gdb(self):
+        """Test repeated expressions with gdb."""
+        self.buildDefault()
+        self.run_gdb_repeated_exprs()
+
+    def run_lldb_repeated_exprs(self):
+        print "running "+self.testMethodName
+        print "benchmarks result for "+self.testMethodName
+
+    def run_gdb_repeated_exprs(self):
+        print "running "+self.testMethodName
+        print "benchmarks result for "+self.testMethodName
+
+if __name__ == '__main__':
+    import atexit
+    lldb.SBDebugger.Initialize()
+    atexit.register(lambda: lldb.SBDebugger.Terminate())
+    unittest2.main()

Added: lldb/trunk/test/benchmarks/example/main.c
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/benchmarks/example/main.c?rev=136553&view=auto
==============================================================================
--- lldb/trunk/test/benchmarks/example/main.c (added)
+++ lldb/trunk/test/benchmarks/example/main.c Fri Jul 29 20:39:58 2011
@@ -0,0 +1,6 @@
+#include <stdio.h>
+
+int main(int argc, char const *argv[]) {
+    printf("Hello world.\n");
+    return 0;
+}

Modified: lldb/trunk/test/dotest.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/dotest.py?rev=136553&r1=136552&r2=136553&view=diff
==============================================================================
--- lldb/trunk/test/dotest.py (original)
+++ lldb/trunk/test/dotest.py Fri Jul 29 20:39:58 2011
@@ -70,6 +70,9 @@
 # By default, both command line and Python API tests are performed.
 just_do_python_api_test = False
 
+# By default, benchmarks tests are not run.
+just_do_benchmarks_test = False
+
 # The blacklist is optional (-b blacklistFile) and allows a central place to skip
 # testclass's and/or testclass.testmethod's.
 blacklist = None
@@ -162,6 +165,8 @@
        use @python_api_test to decorate a test case as lldb Python API test
 +a   : just do lldb Python API tests
        do not specify both '-a' and '+a' at the same time
++b   : just do benchmark tests
+       use @benchmark_test to decorate a test case as such
 -b   : read a blacklist file specified after this option
 -c   : read a config file specified after this option
        the architectures and compilers (note the plurals) specified via '-A' and '-C'
@@ -287,6 +292,7 @@
 
     global dont_do_python_api_test
     global just_do_python_api_test
+    global just_do_benchmarks_test
     global blacklist
     global blacklistConfig
     global configFile
@@ -347,6 +353,9 @@
         elif sys.argv[index].startswith('+a'):
             just_do_python_api_test = True
             index += 1
+        elif sys.argv[index].startswith('+b'):
+            just_do_benchmarks_test = True
+            index += 1
         elif sys.argv[index].startswith('-b'):
             # Increment by 1 to fetch the blacklist file name option argument.
             index += 1
@@ -815,6 +824,7 @@
 # Put dont/just_do_python_api_test in the lldb namespace, too.
 lldb.dont_do_python_api_test = dont_do_python_api_test
 lldb.just_do_python_api_test = just_do_python_api_test
+lldb.just_do_benchmarks_test = just_do_benchmarks_test
 
 # Turn on lldb loggings if necessary.
 lldbLoggings()

Modified: lldb/trunk/test/lldbtest.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/lldbtest.py?rev=136553&r1=136552&r2=136553&view=diff
==============================================================================
--- lldb/trunk/test/lldbtest.py (original)
+++ lldb/trunk/test/lldbtest.py Fri Jul 29 20:39:58 2011
@@ -240,7 +240,7 @@
     def wrapper(self, *args, **kwargs):
         try:
             if lldb.dont_do_python_api_test:
-                self.skipTest("Skip Python API tests")
+                self.skipTest("python api tests")
         except AttributeError:
             pass
         return func(self, *args, **kwargs)
@@ -249,6 +249,24 @@
     wrapper.__python_api_test__ = True
     return wrapper
 
+from functools import wraps
+def benchmarks_test(func):
+    """Decorate the item as a benchmarks test."""
+    if isinstance(func, type) and issubclass(func, unittest2.TestCase):
+        raise Exception("@benchmarks_test can only be used to decorate a test method")
+    @wraps(func)
+    def wrapper(self, *args, **kwargs):
+        try:
+            if not lldb.just_do_benchmarks_test:
+                self.skipTest("benchmarks tests")
+        except AttributeError:
+            pass
+        return func(self, *args, **kwargs)
+
+    # Mark this function as such to separate them from the regular tests.
+    wrapper.__benchmarks_test__ = True
+    return wrapper
+
 class recording(StringIO.StringIO):
     """
     A nice little context manager for recording the debugger interactions into
@@ -514,7 +532,20 @@
                 if getattr(testMethod, "__python_api_test__", False):
                     pass
                 else:
-                    self.skipTest("Skip lldb command line test")
+                    self.skipTest("non python api test")
+        except AttributeError:
+            pass
+
+        # Benchmarks test is decorated with @benchmarks_test,
+        # which also sets the "__benchmarks_test__" attribute of the
+        # function object to True.
+        try:
+            if lldb.just_do_benchmarks_test:
+                testMethod = getattr(self, self._testMethodName)
+                if getattr(testMethod, "__benchmarks_test__", False):
+                    pass
+                else:
+                    self.skipTest("non benchmarks test")
         except AttributeError:
             pass
 

Modified: lldb/trunk/test/python_api/lldbutil/frame/TestFrameUtils.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/python_api/lldbutil/frame/TestFrameUtils.py?rev=136553&r1=136552&r2=136553&view=diff
==============================================================================
--- lldb/trunk/test/python_api/lldbutil/frame/TestFrameUtils.py (original)
+++ lldb/trunk/test/python_api/lldbutil/frame/TestFrameUtils.py Fri Jul 29 20:39:58 2011
@@ -18,6 +18,7 @@
         self.line = line_number('main.c',
                                 "// Find the line number here.")
 
+    @python_api_test
     def test_frame_utils(self):
         """Test utility functions for the frame object."""
         self.buildDefault()

Modified: lldb/trunk/test/python_api/lldbutil/iter/TestLLDBIterator.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/python_api/lldbutil/iter/TestLLDBIterator.py?rev=136553&r1=136552&r2=136553&view=diff
==============================================================================
--- lldb/trunk/test/python_api/lldbutil/iter/TestLLDBIterator.py (original)
+++ lldb/trunk/test/python_api/lldbutil/iter/TestLLDBIterator.py Fri Jul 29 20:39:58 2011
@@ -19,16 +19,19 @@
         self.line1 = line_number('main.cpp', '// Set break point at this line.')
         self.line2 = line_number('main.cpp', '// And that line.')
 
+    @python_api_test
     def test_lldb_iter_module(self):
         """Test module_iter works correctly for SBTarget -> SBModule."""
         self.buildDefault()
         self.lldb_iter_module()
 
+    @python_api_test
     def test_lldb_iter_breakpoint(self):
         """Test breakpoint_iter works correctly for SBTarget -> SBBreakpoint."""
         self.buildDefault()
         self.lldb_iter_breakpoint()
 
+    @python_api_test
     def test_lldb_iter_frame(self):
         """Test iterator works correctly for SBProcess->SBThread->SBFrame."""
         self.buildDefault()

Modified: lldb/trunk/test/python_api/lldbutil/iter/TestRegistersIterator.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/python_api/lldbutil/iter/TestRegistersIterator.py?rev=136553&r1=136552&r2=136553&view=diff
==============================================================================
--- lldb/trunk/test/python_api/lldbutil/iter/TestRegistersIterator.py (original)
+++ lldb/trunk/test/python_api/lldbutil/iter/TestRegistersIterator.py Fri Jul 29 20:39:58 2011
@@ -18,6 +18,7 @@
         # Find the line number to break inside main().
         self.line1 = line_number('main.cpp', '// Set break point at this line.')
 
+    @python_api_test
     def test_iter_registers(self):
         """Test iterator works correctly for lldbutil.iter_registers()."""
         self.buildDefault()

Modified: lldb/trunk/test/python_api/lldbutil/process/TestPrintStackTraces.py
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/test/python_api/lldbutil/process/TestPrintStackTraces.py?rev=136553&r1=136552&r2=136553&view=diff
==============================================================================
--- lldb/trunk/test/python_api/lldbutil/process/TestPrintStackTraces.py (original)
+++ lldb/trunk/test/python_api/lldbutil/process/TestPrintStackTraces.py Fri Jul 29 20:39:58 2011
@@ -18,6 +18,7 @@
         # Find the line number to break inside main().
         self.line = line_number('main.cpp', '// Set break point at this line.')
 
+    @python_api_test
     def test_stack_traces(self):
         """Test SBprocess and SBThread APIs with printing of the stack traces."""
         self.buildDefault()





More information about the lldb-commits mailing list