<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Aug 15, 2018, at 12:27 PM, Zachary Turner <<a href="mailto:zturner@google.com" class="">zturner@google.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><div dir="auto" class="">Back to the original proposal, my biggest concern is that a single inline test could generate many FileCheck invocations. This could cause measurable performance impact on the test suite. Have you considered this?</div></div></div></blockquote><div><br class=""></div><div>That's a good point. I hadn't considered that. My thoughts on that are;</div><div><br class=""></div><div>- It's relatively cheap to create a FileCheck process. If the build is (A|T)sanified, we can copy in a non-sanitized FileCheck to speed things up.</div><div><br class=""></div><div>- Based on the time it takes to run check-{llvm,clang} locally, which have ~56,000 FileCheck invocations, my intuition is that the overhead ought to be manageable.</div><div><br class=""></div><div>- The status quo is doing Python's re.search over a chunk of command output. My (unverified) intuition is that FileCheck won't be slower than that. Actually, FileCheck has an algorithmic advantage because it doesn't re-scan the input text from the beginning of the text each time it tries to match a substring. `self.expect` does.</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="auto" class=""><br class=""></div><div dir="auto" class="">Another possible solution is what i mentioned earlier, basically to expose a debugger object model. This would allow you to accomplish what you want without FileCheck, while simultaneously being making many other types of tests easier to write at the same time. On the other hand, it’s a larger effort to create this system, but I think long term it would pay back enormously (it’s even useful as a general purpose debugger feature, not limited to testing)</div></div></blockquote><div><br class=""></div><div>I'd volunteer to work on that. At the moment I really need to get some form of testing put together for my patches soon.</div><div><br class=""></div><div>vedant</div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Tue, Aug 14, 2018 at 5:31 PM Vedant Kumar via lldb-dev <<a href="mailto:lldb-dev@lists.llvm.org" class="">lldb-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space" class="">Hello,<div class=""><br class=""></div><div class="">I'd like to make FileCheck available within lldb inline tests, in addition to existing helpers like 'runCmd' and 'expect'.</div><div class=""><br class=""></div><div class="">My motivation is that several tests I'm working on can't be made as rigorous as they need to be without FileCheck-style checks. In particular, the 'matching', 'substrs', and 'patterns' arguments to runCmd/expect don't allow me to verify the ordering of checked input, to be stringent about line numbers, or to capture & reuse snippets of text from the input stream.</div><div class=""><br class=""></div><div class="">I'd curious to know if anyone else is interested or would be willing to review this (<a href="https://reviews.llvm.org/D50751" target="_blank" class="">https://reviews.llvm.org/D50751</a>).</div><div class=""><br class=""></div><div class="">Here's an example of an inline test which benefits from FileCheck-style checking. This test is trying to check that certain frames appear in a backtrace when stopped inside of the "sink" function. Notice that without FileCheck, it's not possible to verify the order in which frames are printed, and that dealing with line numbers would be cumbersome.</div><div class=""><br class=""></div><div class="">```</div><div class="">--- a/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp<br class="">+++ b/lldb/packages/Python/lldbsuite/test/functionalities/tail_call_frames/unambiguous_sequence/main.cpp<br class="">@@ -9,16 +9,21 @@<br class=""> <br class=""> volatile int x;<br class=""> <br class="">+// CHECK: frame #0: {{.*}}sink() at main.cpp:[[@LINE+2]] [opt]<br class=""> void __attribute__((noinline)) sink() {<br class="">- x++; //% self.expect("bt", substrs = ['main', 'func1', 'func2', 'func3', 'sink'])<br class="">+ x++; //% self.filecheck("bt", "main.cpp")<br class=""> }<br class=""> <br class="">+// CHECK-NEXT: frame #1: {{.*}}func3() {{.*}}[opt] [artificial]<br class=""> void __attribute__((noinline)) func3() { sink(); /* tail */ }<br class=""> <br class="">+// CHECK-NEXT: frame #2: {{.*}}func2() at main.cpp:[[@LINE+1]] [opt]<br class=""> void __attribute__((disable_tail_calls, noinline)) func2() { func3(); /* regular */ }<br class=""> <br class="">+// CHECK-NEXT: frame #3: {{.*}}func1() {{.*}}[opt] [artificial]<br class=""> void __attribute__((noinline)) func1() { func2(); /* tail */ }<br class=""> <br class="">+// CHECK-NEXT: frame #4: {{.*}}main at main.cpp:[[@LINE+2]] [opt]<br class=""> int __attribute__((disable_tail_calls)) main() {<br class=""> func1(); /* regular */<br class=""> return 0;</div><div class="">```</div><div class=""><br class=""></div><div class="">For reference, here's the output of the "bt" command:</div><div class=""><br class=""></div><div class="">```</div><div class="">runCmd: bt<br class="">output: * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1<br class=""> * frame #0: 0x000000010c6a6f64 a.out`sink() at main.cpp:14 [opt]<br class=""> frame #1: 0x000000010c6a6f70 a.out`func3() at main.cpp:15 [opt] [artificial]<br class=""> frame #2: 0x000000010c6a6f89 a.out`func2() at main.cpp:21 [opt]<br class=""> frame #3: 0x000000010c6a6f90 a.out`func1() at main.cpp:21 [opt] [artificial]<br class=""> frame #4: 0x000000010c6a6fa9 a.out`main at main.cpp:28 [opt]<br class=""></div><div class="">```</div><div class=""><br class=""></div><div class="">thanks,</div><div class="">vedant</div></div>_______________________________________________<br class="">
lldb-dev mailing list<br class="">
<a href="mailto:lldb-dev@lists.llvm.org" target="_blank" class="">lldb-dev@lists.llvm.org</a><br class="">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev</a><br class="">
</blockquote></div></div>
</div></blockquote></div><br class=""></body></html>