[PATCH] D148854: [llvm-stress] Switch to a FuzzMutate driver

[Peter Rong via Phabricator via llvm-commits]

Peter added a comment.

In D148854#4288065 <https://reviews.llvm.org/D148854#4288065>, @bjope wrote:

> In D148854#4287879 <https://reviews.llvm.org/D148854#4287879>, @Peter wrote:
>
>> In D148854#4286441 <https://reviews.llvm.org/D148854#4286441>, @bjope wrote:
>>
>>> In D148854#4286416 <https://reviews.llvm.org/D148854#4286416>, @fhahn wrote:
>>>
>>>> I think this makes sense in general, but I think @bjope is using `llvm-stress` heavily IIRC, so it would be good for them to try the new version to see if there are any obvious issues.
>>>>
>>>> Also, it would be good to have at least some basic tests.
>>>
>>> I think "heavily" is a bit of an overstatement. But we do use llvm-stress as part of validation of "main" before updating our downstream branches.
>>> I'd be happy to try using this patch, but not sure I'll find time doing it already today.
>>
>> Hi @bjope, we are developing FuzzMutate here and are very interested in how you use `llvm-stress`. Can you elaborate more on " validation of "main" "?
>>
>> P.S> We are using FuzzMutate to validate backend of LLVM (`llc`)
>
> Ok, "validation of main" was a bit vague (and I realize that it wasn't the full story either).
>
> I'm involved in a project where we use LLVM for an out-of-tree target. We update out downstream branches on a regular basis, by merging new commits from the llvm-project main branch (that is why I referred to "main").
>
> Before we merge a new commit from upstream (main) we run some tests using in-tree targets on llvm-project's main branch. This is done as a kind of quick validation that the commit we want to merge isn't total crap, and one part of that validation is using llvm-stress. Partially that can be seen as a to verify that llvm-stress is working, rather than validating that for example the backend isn't broken.
> More practically about what we do is that we have a script that generates a number of LLVM IR programs using llvm-stress. And then we run llc on those programs to see that there are no crashes (we enable some backend verifiers such as the machine verifier when doing these tests). So nothing fancy really. We've been doing this for a long time just to get some extra testing, and we still run those tests even if we rarely see any problems in these kinds of tests.
>
> What I didn't mention earlier is that we also use llvm-stress as a fuzzy test program generator when validating our out-of-tree target. So that is one reason why we want to see that llvm-stress amd llc work nicely together already when running same kinds of tests on main branch with in-tree targets.
> The tests for the out-of-tree target are also just simple "do-not-crash" kinds of tests, but the difference is that we use the out-of-tree target in llc.
>
> I think these tests were more important when the out-of-tree target was more immature. In those early stages llvm-stress helped out with getting some code coverage for the backend. Quickly providing lots of different IR to use as input to llc.

Thanks for the explanation, that makes a lot more sense.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D148854/new/

https://reviews.llvm.org/D148854