[Lldb-commits] [PATCH] D24304: Braindump of what an LLDB lit test might look like

[Zachary Turner via lldb-commits]

zturner created this revision.
zturner added subscribers: LLDB, lldb-commits, rnk.

I was trying to brainstorm what a real test that currently exists might look like if converted to a lit-style test.  Not even proposing we do this, just figured I would throw up what I came up with in case anyone wants to see something concrete.  These tests come from lang/cpp/namespace.  There is one `.test` file for each method in each existing `.py` file.  This way everything gets run in parallel.  A couple observations and musings:

1. There is NO MAKEFILE.  Usually this is going to be fine, as most Makefiles are trivial anyway and do nothing.  If we wanted to take this idea further so that it works for our entire test suite, we would need a way to provide custom build settings.  I envision happening through directives in each test file but there are some issues obviously.  We'd have to figure it out.
2. There are no skip / xfail directives.  Again, we could design our own per-file directives for this.
3. There's no manual invocation of the Python APIs.  I actually think this is an advantage.  It makes the tests easier to understand.  Whether this is flexible enough to support everything that LLDB tests do is an open question.  We could always provide a `#script <label>` directive and then embed some Python code below if we need to do something funky.
4. The checks and directives are interspersed with the code.  I think this makes it much easier to look at a test and digest what it does rather than having to click through many different files.  Although sometimes a test is specifically testing something that has to do with multiple object files.  I didn't try to address that here, but again, with the freedom to design whatever set of directives we need to, I think it's doable.
5. Source code is reproduced in every file.  This means every test will be compiling its own target.  But in practice I think this will still be a speedup over our current state of affairs, where every test STILL recompiles its own target, but it does so serially.  In the solution here, at least the compiler would be run in parallel for each test.

Again, this is just some very early high-level thoughts about what a test might look like.  I mostly approached this from the angle of "what would allow this test to be the easiest to write and easiest to understand".  

https://reviews.llvm.org/D24304

Files:
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/breakpoints_a_func_full.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/breakpoints_func_auto.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/breakpoints_func_full.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/file_scope_lookup_with_run_cmd.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/function_scope_lookup_with_run_cmd.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/scope_after_using_declaration_lookup_with_run_cmd.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/scope_after_using_directive_lookup_with_run_cmd.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/scope_ambiguity_after_using_lookup_with_run_cmd.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/scope_lookup_before_using_with_run_cmd.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/scope_lookup_shadowed_by_using_with_run_cmd.test
  packages/Python/lldbsuite/test/lang/cpp/namespace_lit/scope_lookup_with_run_cmd.test

Index: packages/Python/lldbsuite/test/lang/cpp/namespace_lit/scope_lookup_with_run_cmd.test
===================================================================
--- /dev/null
+++ packages/Python/lldbsuite/test/lang/cpp/namespace_lit/scope_lookup_with_run_cmd.test
@@ -0,0 +1,63 @@
+; int func() { return 42; }
+; namespace A {
+;   int func(int N) { return N + 3; }
+;   int func2() { return 42; }
+;   namespace B {
+;     int func() { return 4; }
+;   }
+; }
+; 
+; void exec() {
+;  #breakpoint
+;  #verify("expr -- func()", start="(int) [[%PH%]] = 1")
+;  #verify("expr -- A::B::func()", start="(int) [[%PH%]] = 4")
+;  #verify("expr -- func(10)", start="(int) [[%PH%]] = 11")
+;  #verify("expr -- ::func()", start="(int) [[%PH%]] = 1")
+;  #verify("expr -- A::foo()", start="(int) [[%PH%]] = 42")
+;  #continue
+; }
+; namespace A {
+;   void exec() {
+;     #breakpoint
+;     #verify("expr -- func(10)", start="(int) [[%PH%]] = 13")
+;     #verify("expr -- B::func()", start="(int) [[%PH%]] = 4")
+;     #verify("expr -- func()", start="(int) [[%PH%]] = 3")
+;     #continue
+;   }
+;   namespace B {
+;     void exec() {
+;       #breakpoint
+;       #verify("expr -- func()", start="(int) $8 = 4")
+;       #verify("expr -- A::func()", start="(int) $9 = 3")
+;       #verify("expr -- func(10)", start="(int) $10 = 13")
+;       #continue
+;     }
+;     void exec2() {
+;       using A::func;
+;       #breakpoint
+;       #verify("expr -- A::func(10)", start="(int) $11 = 13")
+;       #continue
+;     }
+;   }
+; }
+; void exec_before_using() {
+;   #breakpoint
+;   #verify("expr -- ::func()", start="(int) $12 = 1")
+;   #verify("expr -- B::func()", start="(int) $13 = 4")
+;   #continue
+; }
+; void exec_after_using() {
+;   #breakpoint
+;   #verify("expr -- ::func()", start="(int) $14 = 1")
+;   #verify("expr -- B::func()", start="(int) $15 = 4")
+;   #continue
+; }
+;
+; void LIT_RUN_TEST() {
+;   exec();
+;   A::exec();
+;   A::B::exec();
+;   A::B::exec2();
+;   exec_before_using();
+;   exec_after_using();
+; }
Index: packages/Python/lldbsuite/test/lang/cpp/namespace_lit/breakpoints_func_auto.test
===================================================================
--- /dev/null
+++ packages/Python/lldbsuite/test/lang/cpp/namespace_lit/breakpoints_func_auto.test
@@ -0,0 +1,18 @@
+; int func() { return 42; }
+; int func(int N) { return 42 + N; }
+; namespace A {
+;   int func(int N) { return N + 3; }
+;   int func() { return 42; }
+;   namespace B {
+;     int func() { return 4; }
+;   }
+; }
+;
+; void LIT_RUN_TEST() {
+;   #break_by_name("func")
+;   #ensure_breakpoint("func()")
+;   #ensure_breakpoint("func(int)")
+;   #ensure_breakpoint("A::func()")
+;   #ensure_breakpoint("A::func(int)")
+;   #ensure_breakpoint("A::B::func()")
+; }


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D24304.70560.patch
Type: text/x-patch
Size: 2856 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20160907/2659206e/attachment.bin>