[Lldb-commits] [lldb] r257644 - Fix an issue where scripted commands would not actually print any of their output if an immediate output file was set in the result object via a Python file object

Thu Jan 14 10:58:29 PST 2016

Wouldn't testing with output redirecxted to a file still test that it is
being streamed as it is obtained rather than a big dump at the end?  I mean
that's what stdout is right?  Just a file.  If you use a file on the
filesystem instead, just check the contents of the file after each
operation.

On Thu, Jan 14, 2016 at 10:42 AM Jim Ingham <jingham at apple.com> wrote:

> I worry giving up on testing using Python's stdout for the immediate
> output stream.  This is a very useful feature, allowing users to make
> Python based commands that produce a bunch of output, and stream the output
> as it is being gathered rather than having the command stall and then dump
> a bunch of text at the end.  This isn't speculative, that's how many of the
> commands that the OS X kernel team ships for inspecting kernel state work.
>
> So we really should be testing this feature.  Maybe the flakiness on Linux
> is just a pexpect bug, and streaming to stdout would always work correctly
> hooked up to a "real" terminal.  But until we know that we should presume
> that it is something in LLDB->Python or in the way we use Python, and keep
> testing it.
>
> If you want to separate the two issues, then it would be fine to write
> another test that just tests the type maps for FILE *, but I still think
> this one is valuable.
>
> Jim
>
> > On Jan 14, 2016, at 10:16 AM, Zachary Turner via lldb-commits <
> lldb-commits at lists.llvm.org> wrote:
> >
> > How much time do you think it would take to determine whether or not
> using the file-based approach would work?  Because on the surface it sounds
> fairly straightforward, and fixing it that way would allow us to not have
> to xfail this on more platforms for reasons that we don't understand.
> >
> > On Thu, Jan 14, 2016 at 10:15 AM Enrico Granata via lldb-commits <
> lldb-commits at lists.llvm.org> wrote:
> > The log just shows a timeout is happening in pexpect() - which I don’t
> have a ready explanation for
> >
> > X-failing for now is the proper recourse. But you might want to debug
> this at some point, since it’s weird behavior. It looks like the command is
> not even just silently erroring out - from the log you sent it looked stuck
> somewhere..
> >
> > Is there any chance you can step through and see where the hang is
> happening?
> >
> >> On Jan 14, 2016, at 3:03 AM, Tamas Berghammer <tberghammer at google.com>
> wrote:
> >>
> >> I XFAIL-ed the test for Linux to get the build bot green again and
> filed a bug at https://llvm.org/bugs/show_bug.cgi?id=26139
> >>
> >> On Thu, Jan 14, 2016 at 2:18 AM Ying Chen via lldb-commits <
> lldb-commits at lists.llvm.org> wrote:
> >> Please see attached log file.
> >>
> >> Thanks,
> >> Ying
> >>
> >> On Wed, Jan 13, 2016 at 5:39 PM, Enrico Granata <egranata at apple.com>
> wrote:
> >> From the buildbot log it’s quite hard to tell what could be going on
> >>
> >> Is there any chance you guys could run the test by hand with the “-t
> -v” flags to the dotest.py driver and attach the output of the run?
> >>
> >> That might help figure out where the issue lies
> >>
> >>> On Jan 13, 2016, at 5:35 PM, Ying Chen <chying at google.com> wrote:
> >>>
> >>> Hello Enrico,
> >>>
> >>> The new test has been failing on Ubuntu buildbot. But it's passing on
> some offline Ubuntu machines, I don't understand what caused the difference.
> >>> Could you please help to take a look?
> >>>
> >>>
> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/10299
> >>>
> >>> Thanks,
> >>> Ying
> >>>
> >>> On Wed, Jan 13, 2016 at 11:32 AM, Enrico Granata via lldb-commits <
> lldb-commits at lists.llvm.org> wrote:
> >>>
> >>>> On Jan 13, 2016, at 11:26 AM, Zachary Turner <zturner at google.com>
> wrote:
> >>>>
> >>>> Thanks!  btw would having the command write its output to a file
> instead of stdout solve the pexpect problme as well?
> >>>>
> >>>
> >>> That’s possible - but I would need to play with it a little bit to
> convince myself that it really is a faithful reproduction of my original
> issue
> >>> It’ll take me a little while to get to it - stay tuned.
> >>>
> >>>> On Wed, Jan 13, 2016 at 11:22 AM Enrico Granata <egranata at apple.com>
> wrote:
> >>>>
> >>>>> On Jan 13, 2016, at 10:34 AM, Zachary Turner <zturner at google.com>
> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Wed, Jan 13, 2016 at 10:25 AM Enrico Granata <egranata at apple.com>
> wrote:
> >>>>>> On Jan 13, 2016, at 10:21 AM, Zachary Turner <zturner at google.com>
> wrote:
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Jan 13, 2016 at 10:15 AM Enrico Granata via lldb-commits <
> lldb-commits at lists.llvm.org> wrote:
> >>>>>> +
> >>>>>> +class CommandScriptImmediateOutputTestCase (PExpectTest):
> >>>>>> Does the bug that you were trying to fix occur only when using the
> command_script.py file from the lldb command line?  If you load it from
> within lldb via an LLDB command, does the problem still occur?  If the
> problem you are fixing is not specific to the LLDB command line, I would
> prefer if you write this not as a pexpect test.
> >>>>>
> >>>>> I would love to not touch pexpect :-) But in this case, I can’t see
> a way around it. I am trying to detect whether some text is “physically”
> printed to stdout. And pexpect seems the most obvious straightforward way
> to get that to happen. Note that, in this bug, the result object is filled
> in correctly even if nothing gets printed, so looking at the result won’t
> quite cut it - it really needs to detect output to stdout.
> >>>>> You're calling result.SetImmediateOutputFile(sys.__stdout__).
> Wouldn't it work to use a file on the file system here, and then you open
> that file and look at it after running the commands?  It wouldn't work in
> remote cases, but it already doesn't work on remote cases anyway (as you
> point out below)
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> +
> >>>>>> +    mydir = TestBase.compute_mydir(__file__)
> >>>>>> +
> >>>>>> +    def setUp(self):
> >>>>>> +        # Call super's setUp().
> >>>>>> +        PExpectTest.setUp(self)
> >>>>>> +
> >>>>>> +    @skipIfRemote # test not remote-ready llvm.org/pr24813
> >>>>>> +    @expectedFlakeyFreeBSD("llvm.org/pr25172 fails rarely on the
> buildbot")
> >>>>>> +    @expectedFlakeyLinux("llvm.org/pr25172")
> >>>>>> Are these necessary?  The windows one is necessary (but not if you
> change this to not being a pexpect test as I've requested above), but why
> are the other ones necessary?
> >>>>>
> >>>>> Do we support remote pexpect? As for FreeBSD and Linux, they might
> not be necessary, but I’d rather much remove them (or let the relevant
> platform owners) remove them in a separate commit
> >>>>>
> >>>>> No remote pexpect, so the @skipIfRemote probably needs to be there.
> But I think everyone should be checking in tests enabled by default in the
> widest set of environments possible that you aren't completely sure are
> broken.  It should be up to the platform holders to disable broken tests,
> not to enable working tests.  Because it's much easier to notice a broken
> test going in than it is to notice a working test went in disabled (because
> who's going to think to test it out?).
> >>>>
> >>>> This is a fair point. I’ll enable those platforms in a subsequent
> commit ASAP
> >>>>
> >>>>
> >>>> Thanks,
> >>>> - Enrico
> >>>> 📩 egranata@.com ☎️ 27683
> >>>>
> >>>
> >>>
> >>> Thanks,
> >>> - Enrico
> >>> 📩 egranata@.com ☎️ 27683
> >>>
> >>>
> >>> _______________________________________________
> >>> lldb-commits mailing list
> >>> lldb-commits at lists.llvm.org
> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
> >>>
> >>>
> >>
> >>
> >> Thanks,
> >> - Enrico
> >> 📩 egranata@.com ☎️ 27683
> >>
> >>
> >> _______________________________________________
> >> lldb-commits mailing list
> >> lldb-commits at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
> >
> >
> > Thanks,
> > - Enrico
> > 📩 egranata@.com ☎️ 27683
> >
> > _______________________________________________
> > lldb-commits mailing list
> > lldb-commits at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
> > _______________________________________________
> > lldb-commits mailing list
> > lldb-commits at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20160114/b9b0f7ca/attachment-0001.html>