[PATCH] D76829: [lit] Introduce setup and teardown routines

Fri Mar 27 10:21:10 PDT 2020

ldionne added a comment.

In D76829#1943847 <https://reviews.llvm.org/D76829#1943847>, @broadwaylamb wrote:

> In D76829#1943659 <https://reviews.llvm.org/D76829#1943659>, @ldionne wrote:
>
> > In D76829#1943483 <https://reviews.llvm.org/D76829#1943483>, @broadwaylamb wrote:
> >
> > > @ldionne I tried to find a way but couldn't. All the code in executor classes is run either at configuration time (when we parse `lit.cfg`), or while running each test. There's no way to tell it to run "when all tests have completed", AFAICT.
> >
> >
> > The executor does receive file dependencies. It could copy those and remove them after each test, no? I'm not saying setup/teardown isn't useful in lit, I'm just trying to see what other ways we can solve that problem.
>
>
> It probably could. However, it would be unnecessary work, leading to slowing down test execution time significantly. Copying the inputs involves tarring them, creating a temporary directory on the target, scp'ing the archive to that temporary directory and untarring it. For me, it can //easily// take ~10 seconds (yeah, my target machine is on another hemisphere, but still). And we have 134 std::filesystem tests, that's ~22 minutes spent //only// on copying the inputs to the target. And I haven't even counted the cleanup time.

FWIW, that's needed for correctness because tests could modify these files. When running locally, we would also need to copy the files to temporary directories, but we don't do it.

I'm curious to know -- what are you tarring up exactly? We have only a few small inputs, don't we?

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D76829/new/

https://reviews.llvm.org/D76829