[PATCH] D32048: ClamAV: Copy zlib into ClamAV benchmark

[Kristof Beyls via Phabricator via llvm-commits]

kristof.beyls added a comment.

> Avoid the external zlib dependency in ClamAV by copying zlib-1.2.11
>  source into the benchmark.
> 
> External dependencies are problematic in benchmarks because:
> 
> They are not compiled with the same compiler/flags as the rest of the benchmarks.
>  They are an additional burden to setup when running the test-suite. While zlib is a really popular and ubiquitous library it is still sometimes missing in cross-compilation settings.
>  No external dependencies simplifies the buildsystem.

I agree that it's a good thing to have fewer external dependencies. However I wonder if the alternative to rip out the parts of clamav that need the zlib library would work too, as a way to get rid of this external dependency?
It probably would mean converting the test inputs to not be compressed?
If a non-trivial amount of time is spent in zlib for clamav and we would want to retain benchmarking of the zlib library, maybe it'd be better to add zlib (with appropriate test input) as a separate benchmark to the test-suite?
We have modified other programs in the test-suite before to make them more useful for compiler performance evaluation and tracking, so I think that doing this would not go beyond what we've done before.
My guess is that performance tracking would be slightly easier if zlib was tested separately.

> This will unfortunately increase the overal compilatime of ClamAV and may therefore disrupt the history of CTMark data.

I think it's pretty accepted that if we have a good reason to change the test-suite, we'll do so even if it breaks historical comparison.
The only way to do guaranteed like-for-like comparisons is to keep the version of the test-suite stable in your experiments.


Repository:
  rL LLVM

https://reviews.llvm.org/D32048