[Lldb-commits] [lldb] r116865 - in /lldb/trunk/docs/testsuite: 2010-10-19-14:10:49.059609/ 2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log a-detailed-walkthrough.txt

Johnny Chen johnny.chen at apple.com
Tue Oct 19 14:46:48 PDT 2010

Author: johnny
Date: Tue Oct 19 16:46:48 2010
New Revision: 116865

URL: http://llvm.org/viewvc/llvm-project?rev=116865&view=rev
Finish the first draft of a detailed walkthrough of an lldb test method.


Added: lldb/trunk/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/docs/testsuite/2010-10-19-14%3A10%3A49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log?rev=116865&view=auto
--- lldb/trunk/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log (added)
+++ lldb/trunk/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log Tue Oct 19 16:46:48 2010
@@ -0,0 +1,43 @@
+os command: [['/bin/sh', '-c', 'make clean; make']]
+stdout: rm -rf "a.out" "a.out.dSYM"  main.o main.d
+g++ -arch x86_64 -gdwarf-2 -O0   -c -o main.o main.cpp
+g++ -arch x86_64 -gdwarf-2 -O0  main.o -o "a.out"
+/usr/bin/dsymutil  -o "a.out.dSYM" "a.out"
+stderr: None
+retcode: 0
+runCmd: file /Volumes/data/lldb/svn/trunk/test/settings/a.out
+output: Current executable set to '/Volumes/data/lldb/svn/trunk/test/settings/a.out' (x86_64).
+runCmd: settings set target.process.output-path 'stdout.txt'
+runCmd: settings show target.process.output-path
+output: target.process.output-path (string) = 'stdout.txt'
+Expecting start string: target.process.output-path (string) = 'stdout.txt'
+runCmd: run
+output: Process 43533 launched: '/Volumes/data/lldb/svn/trunk/test/settings/a.out' (x86_64)
+runCmd: process kill
+check of return status not required
+runCmd failed!
+error: Process must be launched.
+Traceback (most recent call last):
+  File "/Volumes/data/lldb/svn/trunk/test/settings/TestSettings.py", line 125, in test_set_output_path
+    "'stdout.txt' exists due to target.process.output-path.")
+AssertionError: False is not True : 'stdout.txt' exists due to target.process.output-path.

Modified: lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt?rev=116865&r1=116864&r2=116865&view=diff
--- lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt (original)
+++ lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt Tue Oct 19 16:46:48 2010
@@ -161,4 +161,144 @@
 the command within an interactive lldb session.
 The CURRENT_EXECUTABLE_SET is an assert message defined in the lldbtest module
-so that it can reused from other test modules.
+so that it can be reused from other test modules.
+By default, the runCmd() is going to check the return status of the command
+execution and fails the test if it is not a success.  The assert message, in our
+case CURRENT_EXECUTABLE_SET, is used in the exception printout if this happens.
+There are cases when we don't care about the return status from the command
+execution.  This can be accomplished by passing the keyword argument pair
+'check=False' to the method.
+After the current executable is set, we'll then execute two more commands:
+        # Set the output-path and verify it is set.
+        self.runCmd("settings set target.process.output-path 'stdout.txt'")
+        self.expect("settings show target.process.output-path",
+                    SETTING_MSG("target.process.output-path"),
+            startstr = "target.process.output-path (string) = 'stdout.txt'")
+The first uses the 'settings set' command to set the static setting
+target.process.output-path to be 'stdout.txt', instead of the default
+'/dev/stdout'.  We then immediately issue a 'settings show' command to check
+that, indeed, the setting did take place.  Notice that we use a new method
+expect() to accomplish the task, which in effect issues a runCmd() behind the
+door and grabs the output from the command execution and expects to match the
+start string of the output against what we pass in as the value of the keyword
+argument pair:
+            startstr = "target.process.output-path (string) = 'stdout.txt'"
+Take a look at TestBase.expect() within lldbtest.py for more details.  Among
+other things, it can also match against a list of regexp patterns as well as a
+list of sub strings.  And it can also perform negative matching, i.e., instead
+of expecting something from the output of command execution, it can perform the
+action of 'not expecting' something.
+This will launch/run the program:
+        self.runCmd("run", RUN_SUCCEEDED)
+And this asserts that the file 'stdout.txt' should be present after running the
+        # The 'stdout.txt' file should now exist.
+        self.assertTrue(os.path.isfile("stdout.txt"),
+                        "'stdout.txt' exists due to target.process.output-path.")
+Also take a look at main.cpp which emits some message to the stdout.  Now, if we
+pass this assertion, it's time to examine the contents of the file to make sure
+it contains the same message as programmed in main.cpp:
+        # Read the output file produced by running the program.
+        with open('stdout.txt', 'r') as f:
+            output = f.read()
+        self.expect(output, exe=False,
+            startstr = "This message should go to standard out.")
+We open the file and read its contents into output, then issue an expect()
+method.  The 'exe=False' keyword argument pair tells expect() that don't try to
+execute the first arg as a command at all.  Instead, treat it as a string to
+match against whatever is thrown in as keyword argument pairs!
+There are also other test methods present in the TestSettings.py mode:
+test_set_prompt(), test_set_term_width(), test_set_auto_confirm(),
+test_with_dsym(), and test_with_dwarf().  We are using the default test loader
+from unittest framework, which uses the 'test' method name prefix to identify
+test methods automatically.
+This finishes the walkthrough of the test method test_set_output_path(self).
+Before we say goodbye, notice the little method definition at the top of the
+    @classmethod
+    def classCleanup(cls):
+        system(["/bin/sh", "-c", "rm -f output.txt"])
+        system(["/bin/sh", "-c", "rm -f stdout.txt"])
+This is a classmethod (as shown by the @classmethod decorator) which allows the
+individual test class to perform cleanup actions after the test harness finishes
+with the particular test class.  This is part of the so-called test fixture in
+the unittest framework.  From http://docs.python.org/library/unittest.html:
+A test fixture represents the preparation needed to perform one or more tests,
+and any associate cleanup actions. This may involve, for example, creating
+temporary or proxy databases, directories, or starting a server process.
+The TestBase class uses such fixture with setUp(self), tearDown(self),
+setUpClass(cls), and tearDownClass(cls).  And within teraDownClass(cls), it
+checks whether the current class has an attribute named 'classCleanup', and
+executes as a method if present.  In this particular case, the classCleanup()
+calls a utility function system() defined in lldbtest.py in order to remove the
+files created by running the program as the tests are executed.
+This system() function uses the Python subprocess module to spawn the process
+and to retrieve its results.  If the test instance passes the keyword argument
+pair 'sender=self', the detailed command execution through the operating system
+also gets recorded in a session object.  If the test instance fails or errors,
+the session info automatically gets dumped to a file grouped under a directory
+named after the timestamp of the particular test suite run.
+For simple cases, look for the timestamp directory in the same directory of the
+test driver program dotest.py.  For example, if we comment out the
+ at expectedFailure decorator for TestSettings.py, and then run the test module:
+/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v settings
+Collected 6 tests
+test_set_auto_confirm (TestSettings.SettingsCommandTestCase)
+Test that after 'set auto-confirm true', manual confirmation should not kick in. ... ok
+test_set_output_path (TestSettings.SettingsCommandTestCase)
+Test that setting target.process.output-path for the launched process works. ... FAIL
+test_set_prompt (TestSettings.SettingsCommandTestCase)
+Test that 'set prompt' actually changes the prompt. ... ok
+test_set_term_width (TestSettings.SettingsCommandTestCase)
+Test that 'set term-width' actually changes the term-width. ... ok
+test_with_dsym (TestSettings.SettingsCommandTestCase)
+Test that run-args and env-vars are passed to the launched process. ... ok
+test_with_dwarf (TestSettings.SettingsCommandTestCase)
+Test that run-args and env-vars are passed to the launched process. ... ok
+FAIL: test_set_output_path (TestSettings.SettingsCommandTestCase)
+Test that setting target.process.output-path for the launched process works.
+Traceback (most recent call last):
+  File "/Volumes/data/lldb/svn/trunk/test/settings/TestSettings.py", line 125, in test_set_output_path
+    "'stdout.txt' exists due to target.process.output-path.")
+AssertionError: False is not True : 'stdout.txt' exists due to target.process.output-path.
+Ran 6 tests in 8.219s
+FAILED (failures=1)
+/Volumes/data/lldb/svn/trunk/test $ ls 2010-10-19-14:10:49.059609
+/Volumes/data/lldb/svn/trunk/test $ 
+We get one failure and a timestamp directory 2010-10-19-14:10:49.059609.
+For education purposes, the directory and its contents are reproduced here in
+the same directory as the current file.

More information about the lldb-commits mailing list