[PATCH] D140239: [llvm-stress] Add a mutation fuzzing mode

[Peter Rong via Phabricator via llvm-commits]

Peter added a comment.

In D140239#4002462 <https://reviews.llvm.org/D140239#4002462>, @reames wrote:

> In D140239#4002453 <https://reviews.llvm.org/D140239#4002453>, @Peter wrote:
>
>> In D140239#4002352 <https://reviews.llvm.org/D140239#4002352>, @reames wrote:
>>
>>> In D140239#4002268 <https://reviews.llvm.org/D140239#4002268>, @reames wrote:
>>>
>>>> In D140239#4002246 <https://reviews.llvm.org/D140239#4002246>, @Peter wrote:
>>>>
>>>>> I think another interesting thing you can all is "How much do you want to mutate". `mutateModule` will only change the module once, i.e. insert one instruction, add one function(may be no one is using it).
>>>>>
>>>>> Can we add a command line arg `--num-mutate=` and default to, say 100, so we mutate the module 100 times before we return it.
>>>>>
>>>>> Just a thought.
>>>>
>>>> Good idea.  If you don't mind I'll leave that to a follow up commit.  I also want to add some command line control of which strategies to select from.  Again, in a separate commit.
>>>
>>> After writing this, I wondered how we'd been testing the new mutation strategies.  The answer turned out to be, we weren't.  This will give a means to write tests for individual mutations going forward as well.
>>
>> Sorry I don't understand what do you mean by we are not testing mutation strategies.
>>
>> If you mean unit testing, we actually have hand written modules and try it on strategies in llvm/unittests/FuzzMutate/StrategiesTest.cpp.
>> If you mean more large scale end-to-end testing, I have been using these strategies the whole winter so they should be good.
>
> Oops, my apologies.  I'd glanced at a recent commit and looked for changes in test/.  I had not spotted the unit test.  That's entirely my error.

NP. I'd also start from test/. Its a myth to me why the framework author put them in unittest/ so I have to follow suit when writing new tests :)


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D140239/new/

https://reviews.llvm.org/D140239