[Lldb-commits] [lldb] r116865 - in /lldb/trunk/docs/testsuite: 2010-10-19-14:10:49.059609/ 2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log a-detailed-walkthrough.txt
Johnny Chen
johnny.chen at apple.com
Tue Oct 19 14:46:48 PDT 2010
Author: johnny
Date: Tue Oct 19 16:46:48 2010
New Revision: 116865
URL: http://llvm.org/viewvc/llvm-project?rev=116865&view=rev
Log:
Finish the first draft of a detailed walkthrough of an lldb test method.
Added:
lldb/trunk/docs/testsuite/2010-10-19-14:10:49.059609/
lldb/trunk/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log
Modified:
lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt
Added: lldb/trunk/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/docs/testsuite/2010-10-19-14%3A10%3A49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log?rev=116865&view=auto
==============================================================================
--- lldb/trunk/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log (added)
+++ lldb/trunk/docs/testsuite/2010-10-19-14:10:49.059609/TestSettings.SettingsCommandTestCase.test_set_output_path.log Tue Oct 19 16:46:48 2010
@@ -0,0 +1,43 @@
+
+os command: [['/bin/sh', '-c', 'make clean; make']]
+stdout: rm -rf "a.out" "a.out.dSYM" main.o main.d
+g++ -arch x86_64 -gdwarf-2 -O0 -c -o main.o main.cpp
+g++ -arch x86_64 -gdwarf-2 -O0 main.o -o "a.out"
+/usr/bin/dsymutil -o "a.out.dSYM" "a.out"
+
+stderr: None
+retcode: 0
+
+
+runCmd: file /Volumes/data/lldb/svn/trunk/test/settings/a.out
+output: Current executable set to '/Volumes/data/lldb/svn/trunk/test/settings/a.out' (x86_64).
+
+
+runCmd: settings set target.process.output-path 'stdout.txt'
+output:
+
+runCmd: settings show target.process.output-path
+output: target.process.output-path (string) = 'stdout.txt'
+
+
+Expecting start string: target.process.output-path (string) = 'stdout.txt'
+Matched
+
+runCmd: run
+output: Process 43533 launched: '/Volumes/data/lldb/svn/trunk/test/settings/a.out' (x86_64)
+
+
+FAIL
+
+runCmd: process kill
+check of return status not required
+runCmd failed!
+error: Process must be launched.
+
+
+Traceback (most recent call last):
+ File "/Volumes/data/lldb/svn/trunk/test/settings/TestSettings.py", line 125, in test_set_output_path
+ "'stdout.txt' exists due to target.process.output-path.")
+AssertionError: False is not True : 'stdout.txt' exists due to target.process.output-path.
+
+
Modified: lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt
URL: http://llvm.org/viewvc/llvm-project/lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt?rev=116865&r1=116864&r2=116865&view=diff
==============================================================================
--- lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt (original)
+++ lldb/trunk/docs/testsuite/a-detailed-walkthrough.txt Tue Oct 19 16:46:48 2010
@@ -161,4 +161,144 @@
the command within an interactive lldb session.
The CURRENT_EXECUTABLE_SET is an assert message defined in the lldbtest module
-so that it can reused from other test modules.
+so that it can be reused from other test modules.
+
+By default, the runCmd() is going to check the return status of the command
+execution and fails the test if it is not a success. The assert message, in our
+case CURRENT_EXECUTABLE_SET, is used in the exception printout if this happens.
+
+There are cases when we don't care about the return status from the command
+execution. This can be accomplished by passing the keyword argument pair
+'check=False' to the method.
+
+After the current executable is set, we'll then execute two more commands:
+
+ # Set the output-path and verify it is set.
+ self.runCmd("settings set target.process.output-path 'stdout.txt'")
+ self.expect("settings show target.process.output-path",
+ SETTING_MSG("target.process.output-path"),
+ startstr = "target.process.output-path (string) = 'stdout.txt'")
+
+The first uses the 'settings set' command to set the static setting
+target.process.output-path to be 'stdout.txt', instead of the default
+'/dev/stdout'. We then immediately issue a 'settings show' command to check
+that, indeed, the setting did take place. Notice that we use a new method
+expect() to accomplish the task, which in effect issues a runCmd() behind the
+door and grabs the output from the command execution and expects to match the
+start string of the output against what we pass in as the value of the keyword
+argument pair:
+
+ startstr = "target.process.output-path (string) = 'stdout.txt'"
+
+Take a look at TestBase.expect() within lldbtest.py for more details. Among
+other things, it can also match against a list of regexp patterns as well as a
+list of sub strings. And it can also perform negative matching, i.e., instead
+of expecting something from the output of command execution, it can perform the
+action of 'not expecting' something.
+
+This will launch/run the program:
+
+ self.runCmd("run", RUN_SUCCEEDED)
+
+And this asserts that the file 'stdout.txt' should be present after running the
+program.
+
+ # The 'stdout.txt' file should now exist.
+ self.assertTrue(os.path.isfile("stdout.txt"),
+ "'stdout.txt' exists due to target.process.output-path.")
+
+Also take a look at main.cpp which emits some message to the stdout. Now, if we
+pass this assertion, it's time to examine the contents of the file to make sure
+it contains the same message as programmed in main.cpp:
+
+ # Read the output file produced by running the program.
+ with open('stdout.txt', 'r') as f:
+ output = f.read()
+
+ self.expect(output, exe=False,
+ startstr = "This message should go to standard out.")
+
+We open the file and read its contents into output, then issue an expect()
+method. The 'exe=False' keyword argument pair tells expect() that don't try to
+execute the first arg as a command at all. Instead, treat it as a string to
+match against whatever is thrown in as keyword argument pairs!
+
+There are also other test methods present in the TestSettings.py mode:
+test_set_prompt(), test_set_term_width(), test_set_auto_confirm(),
+test_with_dsym(), and test_with_dwarf(). We are using the default test loader
+from unittest framework, which uses the 'test' method name prefix to identify
+test methods automatically.
+
+This finishes the walkthrough of the test method test_set_output_path(self).
+Before we say goodbye, notice the little method definition at the top of the
+file:
+
+ @classmethod
+ def classCleanup(cls):
+ system(["/bin/sh", "-c", "rm -f output.txt"])
+ system(["/bin/sh", "-c", "rm -f stdout.txt"])
+
+This is a classmethod (as shown by the @classmethod decorator) which allows the
+individual test class to perform cleanup actions after the test harness finishes
+with the particular test class. This is part of the so-called test fixture in
+the unittest framework. From http://docs.python.org/library/unittest.html:
+
+A test fixture represents the preparation needed to perform one or more tests,
+and any associate cleanup actions. This may involve, for example, creating
+temporary or proxy databases, directories, or starting a server process.
+
+The TestBase class uses such fixture with setUp(self), tearDown(self),
+setUpClass(cls), and tearDownClass(cls). And within teraDownClass(cls), it
+checks whether the current class has an attribute named 'classCleanup', and
+executes as a method if present. In this particular case, the classCleanup()
+calls a utility function system() defined in lldbtest.py in order to remove the
+files created by running the program as the tests are executed.
+
+This system() function uses the Python subprocess module to spawn the process
+and to retrieve its results. If the test instance passes the keyword argument
+pair 'sender=self', the detailed command execution through the operating system
+also gets recorded in a session object. If the test instance fails or errors,
+the session info automatically gets dumped to a file grouped under a directory
+named after the timestamp of the particular test suite run.
+
+For simple cases, look for the timestamp directory in the same directory of the
+test driver program dotest.py. For example, if we comment out the
+ at expectedFailure decorator for TestSettings.py, and then run the test module:
+
+/Volumes/data/lldb/svn/trunk/test $ ./dotest.py -v settings
+----------------------------------------------------------------------
+Collected 6 tests
+
+test_set_auto_confirm (TestSettings.SettingsCommandTestCase)
+Test that after 'set auto-confirm true', manual confirmation should not kick in. ... ok
+test_set_output_path (TestSettings.SettingsCommandTestCase)
+Test that setting target.process.output-path for the launched process works. ... FAIL
+test_set_prompt (TestSettings.SettingsCommandTestCase)
+Test that 'set prompt' actually changes the prompt. ... ok
+test_set_term_width (TestSettings.SettingsCommandTestCase)
+Test that 'set term-width' actually changes the term-width. ... ok
+test_with_dsym (TestSettings.SettingsCommandTestCase)
+Test that run-args and env-vars are passed to the launched process. ... ok
+test_with_dwarf (TestSettings.SettingsCommandTestCase)
+Test that run-args and env-vars are passed to the launched process. ... ok
+
+======================================================================
+FAIL: test_set_output_path (TestSettings.SettingsCommandTestCase)
+Test that setting target.process.output-path for the launched process works.
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File "/Volumes/data/lldb/svn/trunk/test/settings/TestSettings.py", line 125, in test_set_output_path
+ "'stdout.txt' exists due to target.process.output-path.")
+AssertionError: False is not True : 'stdout.txt' exists due to target.process.output-path.
+
+----------------------------------------------------------------------
+Ran 6 tests in 8.219s
+
+FAILED (failures=1)
+/Volumes/data/lldb/svn/trunk/test $ ls 2010-10-19-14:10:49.059609
+TestSettings.SettingsCommandTestCase.test_set_output_path.log
+/Volumes/data/lldb/svn/trunk/test $
+
+We get one failure and a timestamp directory 2010-10-19-14:10:49.059609.
+For education purposes, the directory and its contents are reproduced here in
+the same directory as the current file.
More information about the lldb-commits
mailing list