[PATCH] D30156: llvm-mc-fuzzer: add support for assembly

[Kostya Serebryany via Phabricator via llvm-commits]

kcc added inline comments.


================
Comment at: tools/llvm-mc-fuzzer/llvm-mc-fuzzer.cpp:244
   if (Action == AC_Assemble)
-    errs() << "error: -assemble is not implemented\n";
+    return AssembleOneInput(Data, Size);
   else if (Action == AC_Disassemble)
----------------
bcain wrote:
> dsanders wrote:
> > kcc wrote:
> > > I strongly suggest to make this a separate fuzz target instead of using flags. 
> > > Otherwise it'll be harder to automate running this target. 
> > I'm not sure what you mean here. What difficulties are you thinking of?
> > 
> > FWIW, this is in line with my original intent which was to mimic llvm-mc's interface.
> > I strongly suggest to make this a separate fuzz target instead of using flags. 
> 
> I've preserved the original design for llvm-mc-fuzzer, apparently to imitate llvm-mc.
> 
> Pros/cons of the current design:
> - pro: matches llvm-mc
> - pro: changing focus to probe different paths only requires different command line args
> - con: reproducing fuzzer configuration more difficult because it depends on those args
> - con: libFuzzer might see the uncovered feature set as a goal for coverage (that we already know statically it cannot cover).
> 
> For that last one, it's speculation on my part.
> 
> Kostya, would you be satisfied with this as-is or should I decompose it into two fuzzers?  "Harder to automate" consists of "I must make sure that I can deliver the right command line args to the automation feature"?  Or "won't fit well in oss-fuzz" or something else?
> I'm not sure what you mean here. What difficulties are you thinking of?

Imagine an automated system that runs continuous fuzzing (e.g. https://github.com/google/oss-fuzz).
How are you going to tell it to run the same binary with two different flags and to treat those
as two independent entities?
Of course, it's possible to implement support for something like this, but OSS-Fuzz does not and will not support it. 
(because of KISS: https://en.wikipedia.org/wiki/KISS_principle)

When analyzing the code coverage (manually, or automatically) there will be a huge lump of code that is never reached in one mode, i.e. this 2-in-1 bundle will confuse the analysis. 

Finally, at least in libFuzzer, part of the algorithm is linear by the size of the binary (more precisely: number of instrumented blocks) and so this bundled fuzzer will just be burning CPUs with no reason. 


> FWIW, this is in line with my original intent which was to mimic llvm-mc's interface.
Yes, and I objected back then :) 


Repository:
  rL LLVM

https://reviews.llvm.org/D30156