[llvm-dev] [LNT] Tests for web UI/javascript?

Tue Jul 24 10:01:37 PDT 2018

Sorry for the delayed response. Thanks for pointing this review out to me. I’ve tried it out and managed to run the tests using phantomjs and recent firefox (had no luck with 52 esr) with minimal modifications (just to adjust to updated code). Although I’ve also had to use 2.99 versions since I’ve encountered some issues with 3.x which I’m still investigating. I can submit the updated patch once I’ve figured it out.
However, if I understood correctly, this will only cover unit testing, while I’d like to test the UI/page layout as well. At least something basic, like that the page loads, shows expected controls and diffs. Has anyone looked at this side of things or have any experience with testing it? I’ve looked at some frameworks like Galen, but they’ve shared similar selenium/browser/webdriver compatibility quirkiness, although they’ve generally worked fine with the latest browser versions.
By the way, I’m not a web developer myself (seems to be a common theme) so I’m hesitant to make the final call. Any advice/direction in that regard would be appreciated.

From: Kristof Beyls [mailto:Kristof.Beyls at arm.com]
Sent: Friday, June 29, 2018 10:44
To: Chris Matthews <chris.matthews at apple.com>
Cc: Matthias Braun <mbraun at apple.com>; Danila Malyutin <Danila.Malyutin at synopsys.com>; llvm-dev <llvm-dev at lists.llvm.org>; nd <nd at arm.com>
Subject: Re: [llvm-dev] [LNT] Tests for web UI/javascript?

Just like Matthias, I have little experience in web development and am not up to speed on modern strategies to test javascript embedded in a web page.
That being said, when I looked into enabling testing the javascript functionality in LNT about 2 years ago, I came up with https://reviews.llvm.org/D20161<https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D20161&d=DwMGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=VEV8gWVf26SDOqiMtTxnBloZmItAauQlSqznsCc0KxY&m=QGwEy_CtmEFSa7T6I2Kw8m0dtH53Gsge-xy32iBwiyk&s=QdaP0tUsNaXsWf8aAzsshi0fPDIiiq7H8qnzf_xqMzI&e=> that Chris pointed to.
I think I’ve listed all my reasons for going with Jasmine as a framework on that review, so please have a look there.
The summary is that jasmine (at least at the time) seemed amongst the more popular javascript testing frameworks and that it was the only one that through the jasmine-py package had some integration with python. That integration makes it easy to integrate it with lit (the unit testing driver the rest of LNT uses), and makes installing the needed dependencies easy in the python-centric world LNT lives in.

After getting one issue fixed in jasmine-py (see review for details), I remember I found that the tests could be run, but had some troubles with some combinations of web browsers (see last comment on the review).

I’m afraid I’ve been kept busy with lots of other stuff since, so haven’t followed up since then.

It’s clear there is a need for unit testing the javascript, so my feel is that we should press ahead with getting a way to unit test the javascript functionality, even if running the unit tests cannot be done on all combinations of OS/web browser (due to bugs in the packages jasmine-py relies on).

Danila: if you’d like to review/try out/improve/make D20161 committable: I’d very much welcome that!
I think a major first step to achieve that would be to try out D20161 on your system and check which browsers it works with and which ones it doesn’t. We could document the combinations we’ve found to work and which ones we’ve found to have issues.

Thanks,

Kristof

On 28 Jun 2018, at 23:30, Chris Matthews <chris.matthews at apple.com<mailto:chris.matthews at apple.com>> wrote:

There was this PR to use Jasmine:

https://reviews.llvm.org/D20161<https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D20161&d=DwMGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=VEV8gWVf26SDOqiMtTxnBloZmItAauQlSqznsCc0KxY&m=QGwEy_CtmEFSa7T6I2Kw8m0dtH53Gsge-xy32iBwiyk&s=QdaP0tUsNaXsWf8aAzsshi0fPDIiiq7H8qnzf_xqMzI&e=>

But I think it did not make it into tree.  Kristof, any update on the issues you were encountering? We could move forward with that.  I think we were pretty agnostic to which framework to use as long as it was not hard to install.

JS Testing would be *really* good. I have debugged a lot of issues recently that ended up in the JS.

On Jun 28, 2018, at 10:35 AM, Matthias Braun via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

For checking HTML output: There are some tests in lnt/tests/server/ui/V4Pages.py (and similar) that look at the produced html output. I also recommend installing pytidylib and running the testsuite with --use-tidylib to catch additional html errors.

I don't think we have anything testing the javascript parts, would be nice to see some additional test coverage there[1].

- Matthias

[1] I personally am a compiler developer with little experience in web development. I don't know what strategies exist to test javascript embedded in a webpage...

On Jun 28, 2018, at 9:48 AM, Danila Malyutin via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

Hello.
We’ve started using LNT and found it very handy. However, we found that LNT web UI is missing some features to help our use-cases.
We’d like to add them (and already started doing so) but it’s hard to add them without breaking existing features/workflows because while LNT has some python CLI test coverage the dynamic Web part (mainly JS UI) is not really tested.
So our question is – are there any plans to introduce testing to this part of LNT? Were there any discussions on this topic? Maybe the LNT developers has already decided on the testing framework or have some candidates in mind?
We’d want to help implement this so it’s important to use the same tools.

Thanks.
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=VEV8gWVf26SDOqiMtTxnBloZmItAauQlSqznsCc0KxY&m=QGwEy_CtmEFSa7T6I2Kw8m0dtH53Gsge-xy32iBwiyk&s=wRLY68n17e9H2KNJdmq1XZE1yAUuMENdzjPw8nYnI8s&e=>

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMGaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=VEV8gWVf26SDOqiMtTxnBloZmItAauQlSqznsCc0KxY&m=QGwEy_CtmEFSa7T6I2Kw8m0dtH53Gsge-xy32iBwiyk&s=wRLY68n17e9H2KNJdmq1XZE1yAUuMENdzjPw8nYnI8s&e=>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180724/6577b6d7/attachment-0001.html>