[llvm-dev] [LNT] Tests for web UI/javascript?

Wed Aug 1 13:56:19 PDT 2018

I don’t really have much to add.  I noticed requests-html is using pyppeteer for page rendering. That is chrome only and does not directly provide a testing interface.  I’d be hesitant to resort to any image diffing.

I’d be interested to hear what else is out there.  I know some people like Selenium.   

> On Jul 24, 2018, at 10:01 AM, Danila Malyutin via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> Sorry for the delayed response. Thanks for pointing this review out to me. I’ve tried it out and managed to run the tests using phantomjs and recent firefox (had no luck with 52 esr) with minimal modifications (just to adjust to updated code). Although I’ve also had to use 2.99 versions since I’ve encountered some issues with 3.x which I’m still investigating. I can submit the updated patch once I’ve figured it out.
> However, if I understood correctly, this will only cover unit testing, while I’d like to test the UI/page layout as well. At least something basic, like that the page loads, shows expected controls and diffs. Has anyone looked at this side of things or have any experience with testing it? I’ve looked at some frameworks like Galen, but they’ve shared similar selenium/browser/webdriver compatibility quirkiness, although they’ve generally worked fine with the latest browser versions.
> By the way, I’m not a web developer myself (seems to be a common theme) so I’m hesitant to make the final call. Any advice/direction in that regard would be appreciated.
>  
> From: Kristof Beyls [mailto:Kristof.Beyls at arm.com] 
> Sent: Friday, June 29, 2018 10:44
> To: Chris Matthews <chris.matthews at apple.com>
> Cc: Matthias Braun <mbraun at apple.com>; Danila Malyutin <Danila.Malyutin at synopsys.com>; llvm-dev <llvm-dev at lists.llvm.org>; nd <nd at arm.com>
> Subject: Re: [llvm-dev] [LNT] Tests for web UI/javascript?
>  
> Just like Matthias, I have little experience in web development and am not up to speed on modern strategies to test javascript embedded in a web page.
> That being said, when I looked into enabling testing the javascript functionality in LNT about 2 years ago, I came up with https://reviews.llvm.org/D20161 that Chris pointed to.
> I think I’ve listed all my reasons for going with Jasmine as a framework on that review, so please have a look there.
> The summary is that jasmine (at least at the time) seemed amongst the more popular javascript testing frameworks and that it was the only one that through the jasmine-py package had some integration with python. That integration makes it easy to integrate it with lit (the unit testing driver the rest of LNT uses), and makes installing the needed dependencies easy in the python-centric world LNT lives in.
>  
> After getting one issue fixed in jasmine-py (see review for details), I remember I found that the tests could be run, but had some troubles with some combinations of web browsers (see last comment on the review).
>  
> I’m afraid I’ve been kept busy with lots of other stuff since, so haven’t followed up since then.
>  
> It’s clear there is a need for unit testing the javascript, so my feel is that we should press ahead with getting a way to unit test the javascript functionality, even if running the unit tests cannot be done on all combinations of OS/web browser (due to bugs in the packages jasmine-py relies on).
>  
> Danila: if you’d like to review/try out/improve/make D20161 committable: I’d very much welcome that!
> I think a major first step to achieve that would be to try out D20161 on your system and check which browsers it works with and which ones it doesn’t. We could document the combinations we’ve found to work and which ones we’ve found to have issues.
>  
> Thanks,
>  
> Kristof
>  
> On 28 Jun 2018, at 23:30, Chris Matthews <chris.matthews at apple.com> wrote:
>  
> There was this PR to use Jasmine: 
>  
> https://reviews.llvm.org/D20161
>  
> But I think it did not make it into tree.  Kristof, any update on the issues you were encountering? We could move forward with that.  I think we were pretty agnostic to which framework to use as long as it was not hard to install.
>  
> JS Testing would be *really* good. I have debugged a lot of issues recently that ended up in the JS.
>  
>  
> 
> 
> On Jun 28, 2018, at 10:35 AM, Matthias Braun via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>  
> For checking HTML output: There are some tests in lnt/tests/server/ui/V4Pages.py (and similar) that look at the produced html output. I also recommend installing pytidylib and running the testsuite with --use-tidylib to catch additional html errors. 
>  
> I don't think we have anything testing the javascript parts, would be nice to see some additional test coverage there[1].
>  
> - Matthias
>  
>  
> [1] I personally am a compiler developer with little experience in web development. I don't know what strategies exist to test javascript embedded in a webpage...
> 
> 
> On Jun 28, 2018, at 9:48 AM, Danila Malyutin via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>  
> Hello. 
> We’ve started using LNT and found it very handy. However, we found that LNT web UI is missing some features to help our use-cases.
> We’d like to add them (and already started doing so) but it’s hard to add them without breaking existing features/workflows because while LNT has some python CLI test coverage the dynamic Web part (mainly JS UI) is not really tested.
> So our question is – are there any plans to introduce testing to this part of LNT? Were there any discussions on this topic? Maybe the LNT developers has already decided on the testing framework or have some candidates in mind?
> We’d want to help implement this so it’s important to use the same tools.
> 
> Thanks.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>  
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>  
>  
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3411 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180801/8f6bd172/attachment.bin>