[llvm] r281007 - [libFuzzer] remove use_traces=1 since use_value_profile seems to be strictly better
Kostya Serebryany via llvm-commits
llvm-commits at lists.llvm.org
Thu Sep 8 18:17:03 PDT 2016
Author: kcc
Date: Thu Sep 8 20:17:03 2016
New Revision: 281007
URL: http://llvm.org/viewvc/llvm-project?rev=281007&view=rev
Log:
[libFuzzer] remove use_traces=1 since use_value_profile seems to be strictly better
Removed:
llvm/trunk/lib/Fuzzer/test/fuzzer-traces.test
Modified:
llvm/trunk/docs/LibFuzzer.rst
llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
llvm/trunk/lib/Fuzzer/FuzzerFlags.def
llvm/trunk/lib/Fuzzer/FuzzerInternal.h
llvm/trunk/lib/Fuzzer/FuzzerTraceState.cpp
llvm/trunk/lib/Fuzzer/test/fuzzer-dfsan.test
Modified: llvm/trunk/docs/LibFuzzer.rst
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/LibFuzzer.rst?rev=281007&r1=281006&r2=281007&view=diff
==============================================================================
--- llvm/trunk/docs/LibFuzzer.rst (original)
+++ llvm/trunk/docs/LibFuzzer.rst Thu Sep 8 20:17:03 2016
@@ -240,7 +240,7 @@ The most important command line options
directory. Defaults to 0. This flag can be used to minimize a corpus.
``-minimize_crash``
If 1, minimizes the provided crash input.
- Use with -runs=N or -max_total_time=N to limit the number attempts.
+ Use with -runs=N or -max_total_time=N to limit the number of attempts.
``-reload``
If set to 1 (the default), the corpus directory is re-read periodically to
check for new inputs; this allows detection of new inputs that were discovered
@@ -261,8 +261,6 @@ The most important command line options
blocks are hit; defaults to 1.
``-use_value_profile``
Use `value profile`_ to guide corpus expansion; defaults to 0.
-``-use_traces``
- Use instruction traces (experimental, defaults to 0); see `Data-flow-guided fuzzing`_.
``-only_ascii``
If 1, generate only ASCII (``isprint``+``isspace``) inputs. Defaults to 0.
``-artifact_prefix``
@@ -619,19 +617,6 @@ but there are two downsides.
First, the extra instrumentation may bring up to 2x additional slowdown.
Second, the corpus may grow by several times.
-
-Data-flow-guided fuzzing
-------------------------
-
-*EXPERIMENTAL*.
-With an additional compiler flag ``-fsanitize-coverage=trace-cmp`` (see SanitizerCoverageTraceDataFlow_)
-and extra run-time flag ``-use_traces=1`` the fuzzer will try to apply *data-flow-guided fuzzing*.
-That is, the fuzzer will record the inputs to comparison instructions, switch statements,
-and several libc functions (``memcmp``, ``strcmp``, ``strncmp``, etc).
-It will later use those recorded inputs during mutations.
-
-This mode can be combined with DataFlowSanitizer_ to achieve better sensitivity.
-
Fuzzer-friendly build mode
---------------------------
Sometimes the code under test is not fuzzing-friendly. Examples:
@@ -922,7 +907,6 @@ Trophies
.. _AFL: http://lcamtuf.coredump.cx/afl/
.. _SanitizerCoverage: http://clang.llvm.org/docs/SanitizerCoverage.html
.. _SanitizerCoverageTraceDataFlow: http://clang.llvm.org/docs/SanitizerCoverage.html#tracing-data-flow
-.. _DataFlowSanitizer: http://clang.llvm.org/docs/DataFlowSanitizer.html
.. _AddressSanitizer: http://clang.llvm.org/docs/AddressSanitizer.html
.. _LeakSanitizer: http://clang.llvm.org/docs/LeakSanitizer.html
.. _Heartbleed: http://en.wikipedia.org/wiki/Heartbleed
Modified: llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp?rev=281007&r1=281006&r2=281007&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp (original)
+++ llvm/trunk/lib/Fuzzer/FuzzerDriver.cpp Thu Sep 8 20:17:03 2016
@@ -395,7 +395,6 @@ int FuzzerDriver(int *argc, char ***argv
Options.MutateDepth = Flags.mutate_depth;
Options.UseCounters = Flags.use_counters;
Options.UseIndirCalls = Flags.use_indir_calls;
- Options.UseTraces = Flags.use_traces;
Options.UseMemcmp = Flags.use_memcmp;
Options.UseMemmem = Flags.use_memmem;
Options.ShuffleAtStartUp = Flags.shuffle;
Modified: llvm/trunk/lib/Fuzzer/FuzzerFlags.def
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerFlags.def?rev=281007&r1=281006&r2=281007&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/FuzzerFlags.def (original)
+++ llvm/trunk/lib/Fuzzer/FuzzerFlags.def Thu Sep 8 20:17:03 2016
@@ -43,7 +43,6 @@ FUZZER_FLAG_INT(minimize_crash, 0, "If 1
FUZZER_FLAG_INT(minimize_crash_internal_step, 0, "internal flag")
FUZZER_FLAG_INT(use_counters, 1, "Use coverage counters")
FUZZER_FLAG_INT(use_indir_calls, 1, "Use indirect caller-callee counters")
-FUZZER_FLAG_INT(use_traces, 0, "Experimental: use instruction traces")
FUZZER_FLAG_INT(use_memcmp, 1,
"Use hints from intercepting memcmp, strcmp, etc")
FUZZER_FLAG_INT(use_memmem, 1,
Modified: llvm/trunk/lib/Fuzzer/FuzzerInternal.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerInternal.h?rev=281007&r1=281006&r2=281007&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/FuzzerInternal.h (original)
+++ llvm/trunk/lib/Fuzzer/FuzzerInternal.h Thu Sep 8 20:17:03 2016
@@ -229,7 +229,6 @@ struct FuzzingOptions {
int MutateDepth = 5;
bool UseCounters = false;
bool UseIndirCalls = true;
- bool UseTraces = false;
bool UseMemcmp = true;
bool UseMemmem = true;
bool UseFullCoverageSet = false;
Modified: llvm/trunk/lib/Fuzzer/FuzzerTraceState.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/FuzzerTraceState.cpp?rev=281007&r1=281006&r2=281007&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/FuzzerTraceState.cpp (original)
+++ llvm/trunk/lib/Fuzzer/FuzzerTraceState.cpp Thu Sep 8 20:17:03 2016
@@ -170,7 +170,6 @@ struct TraceBasedMutation {
};
// Declared as static globals for faster checks inside the hooks.
-static bool RecordingTraces = false;
static bool RecordingMemcmp = false;
static bool RecordingMemmem = false;
static bool RecordingValueProfile = false;
@@ -209,9 +208,8 @@ public:
const uint8_t *DesiredData, size_t DataSize);
void StartTraceRecording() {
- if (!Options.UseTraces && !Options.UseMemcmp)
+ if (!Options.UseMemcmp)
return;
- RecordingTraces = Options.UseTraces;
RecordingMemcmp = Options.UseMemcmp;
RecordingMemmem = Options.UseMemmem;
NumMutations = 0;
@@ -220,9 +218,8 @@ public:
}
void StopTraceRecording() {
- if (!RecordingTraces && !RecordingMemcmp)
+ if (!RecordingMemcmp)
return;
- RecordingTraces = false;
RecordingMemcmp = false;
for (size_t i = 0; i < NumMutations; i++) {
auto &M = Mutations[i];
@@ -332,7 +329,7 @@ void TraceState::DFSanCmpCallback(uintpt
uint64_t Arg1, uint64_t Arg2, dfsan_label L1,
dfsan_label L2) {
assert(ReallyHaveDFSan());
- if (!RecordingTraces || !F->InFuzzingThread()) return;
+ if (!F->InFuzzingThread()) return;
if (L1 == 0 && L2 == 0)
return; // Not actionable.
if (L1 != 0 && L2 != 0)
@@ -381,7 +378,7 @@ void TraceState::DFSanSwitchCallback(uin
uint64_t Val, size_t NumCases,
uint64_t *Cases, dfsan_label L) {
assert(ReallyHaveDFSan());
- if (!RecordingTraces || !F->InFuzzingThread()) return;
+ if (!F->InFuzzingThread()) return;
if (!L) return; // Not actionable.
LabelRange LR = GetLabelRange(L);
size_t ValSize = ValSizeInBits / 8;
@@ -451,7 +448,7 @@ int TraceState::TryToAddDesiredData(cons
void TraceState::TraceCmpCallback(uintptr_t PC, size_t CmpSize, size_t CmpType,
uint64_t Arg1, uint64_t Arg2) {
- if (!RecordingTraces || !F->InFuzzingThread()) return;
+ if (!F->InFuzzingThread()) return;
if ((CmpType == ICMP_EQ || CmpType == ICMP_NE) && Arg1 == Arg2)
return; // No reason to mutate.
int Added = 0;
@@ -482,7 +479,7 @@ void TraceState::TraceMemcmpCallback(siz
void TraceState::TraceSwitchCallback(uintptr_t PC, size_t ValSizeInBits,
uint64_t Val, size_t NumCases,
uint64_t *Cases) {
- if (!RecordingTraces || !F->InFuzzingThread()) return;
+ if (F->InFuzzingThread()) return;
size_t ValSize = ValSizeInBits / 8;
bool TryShort = IsTwoByteData(Val);
for (size_t i = 0; i < NumCases; i++)
@@ -512,7 +509,7 @@ void Fuzzer::StopTraceRecording() {
}
void Fuzzer::AssignTaintLabels(uint8_t *Data, size_t Size) {
- if (!Options.UseTraces && !Options.UseMemcmp) return;
+ if (!Options.UseMemcmp) return;
if (!ReallyHaveDFSan()) return;
TS->EnsureDfsanLabels(Size);
for (size_t i = 0; i < Size; i++)
@@ -520,7 +517,7 @@ void Fuzzer::AssignTaintLabels(uint8_t *
}
void Fuzzer::InitializeTraceState() {
- if (!Options.UseTraces && !Options.UseMemcmp) return;
+ if (!Options.UseMemcmp) return;
TS = new TraceState(MD, Options, this);
}
@@ -603,7 +600,6 @@ static void AddValueForSingleVal(void *P
} // namespace fuzzer
using fuzzer::TS;
-using fuzzer::RecordingTraces;
using fuzzer::RecordingMemcmp;
using fuzzer::RecordingValueProfile;
@@ -611,21 +607,11 @@ extern "C" {
void __dfsw___sanitizer_cov_trace_cmp(uint64_t SizeAndType, uint64_t Arg1,
uint64_t Arg2, dfsan_label L0,
dfsan_label L1, dfsan_label L2) {
- if (!RecordingTraces) return;
- assert(L0 == 0);
- uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
- uint64_t CmpSize = (SizeAndType >> 32) / 8;
- uint64_t Type = (SizeAndType << 32) >> 32;
- TS->DFSanCmpCallback(PC, CmpSize, Type, Arg1, Arg2, L1, L2);
}
#define DFSAN_CMP_CALLBACK(N) \
void __dfsw___sanitizer_cov_trace_cmp##N(uint64_t Arg1, uint64_t Arg2, \
dfsan_label L1, dfsan_label L2) { \
- if (RecordingTraces) \
- TS->DFSanCmpCallback( \
- reinterpret_cast<uintptr_t>(__builtin_return_address(0)), N, \
- fuzzer::ICMP_EQ, Arg1, Arg2, L1, L2); \
}
DFSAN_CMP_CALLBACK(1)
@@ -636,9 +622,6 @@ DFSAN_CMP_CALLBACK(8)
void __dfsw___sanitizer_cov_trace_switch(uint64_t Val, uint64_t *Cases,
dfsan_label L1, dfsan_label L2) {
- if (!RecordingTraces) return;
- uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
- TS->DFSanSwitchCallback(PC, Cases[1], Val, Cases[0], Cases+2, L1);
}
void dfsan_weak_hook_memcmp(void *caller_pc, const void *s1, const void *s2,
@@ -750,50 +733,30 @@ void __sanitizer_weak_hook_memmem(void *
__attribute__((visibility("default")))
void __sanitizer_cov_trace_cmp(uint64_t SizeAndType, uint64_t Arg1,
uint64_t Arg2) {
- if (RecordingTraces) {
- uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
- uint64_t CmpSize = (SizeAndType >> 32) / 8;
- uint64_t Type = (SizeAndType << 32) >> 32;
- TS->TraceCmpCallback(PC, CmpSize, Type, Arg1, Arg2);
- }
if (RecordingValueProfile)
fuzzer::AddValueForCmp(__builtin_return_address(0), Arg1, Arg2);
}
-// Adding if(RecordingTraces){...} slows down the VP callbacks.
-// Once we prove that VP is as strong as traces, delete this.
-#define MAYBE_RECORD_TRACE(N) \
- if (RecordingTraces) { \
- uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0)); \
- TS->TraceCmpCallback(PC, N, fuzzer::ICMP_EQ, Arg1, Arg2); \
- }
-
__attribute__((visibility("default")))
void __sanitizer_cov_trace_cmp8(uint64_t Arg1, int64_t Arg2) {
fuzzer::AddValueForCmp(__builtin_return_address(0), Arg1, Arg2);
- MAYBE_RECORD_TRACE(8);
}
__attribute__((visibility("default")))
void __sanitizer_cov_trace_cmp4(uint32_t Arg1, int32_t Arg2) {
fuzzer::AddValueForCmp(__builtin_return_address(0), Arg1, Arg2);
- MAYBE_RECORD_TRACE(4);
}
__attribute__((visibility("default")))
void __sanitizer_cov_trace_cmp2(uint16_t Arg1, int16_t Arg2) {
fuzzer::AddValueForCmp(__builtin_return_address(0), Arg1, Arg2);
- MAYBE_RECORD_TRACE(2);
}
__attribute__((visibility("default")))
void __sanitizer_cov_trace_cmp1(uint8_t Arg1, int8_t Arg2) {
fuzzer::AddValueForCmp(__builtin_return_address(0), Arg1, Arg2);
- MAYBE_RECORD_TRACE(1);
}
__attribute__((visibility("default")))
void __sanitizer_cov_trace_switch(uint64_t Val, uint64_t *Cases) {
- if (!RecordingTraces) return;
- uintptr_t PC = reinterpret_cast<uintptr_t>(__builtin_return_address(0));
- TS->TraceSwitchCallback(PC, Cases[1], Val, Cases[0], Cases + 2);
+ // TODO(kcc): support value profile here.
}
__attribute__((visibility("default")))
Modified: llvm/trunk/lib/Fuzzer/test/fuzzer-dfsan.test
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/fuzzer-dfsan.test?rev=281007&r1=281006&r2=281007&view=diff
==============================================================================
--- llvm/trunk/lib/Fuzzer/test/fuzzer-dfsan.test (original)
+++ llvm/trunk/lib/Fuzzer/test/fuzzer-dfsan.test Thu Sep 8 20:17:03 2016
@@ -4,13 +4,8 @@ CHECK2: BINGO
CHECK3: BINGO
CHECK4: BINGO
-CHECK_DFSanCmpCallback: DFSanCmpCallback: PC
-CHECK_DFSanSwitchCallback: DFSanSwitchCallback: PC
CHECK_DFSanMemcmpCallback: DFSanMemcmpCallback: Pos
-RUN: not LLVMFuzzer-SimpleCmpTest-DFSan -use_traces=1 -seed=1 -runs=10000000 -timeout=5 2>&1 | FileCheck %s --check-prefix=CHECK1
-RUN: LLVMFuzzer-SimpleCmpTest-DFSan -use_traces=1 -seed=1 -runs=100 -timeout=5 -verbosity=3 2>&1 | FileCheck %s -check-prefix=CHECK_DFSanCmpCallback
-
RUN: not LLVMFuzzer-MemcmpTest-DFSan -use_traces=1 -seed=1 -runs=100000 -timeout=5 2>&1 | FileCheck %s --check-prefix=CHECK2
RUN: LLVMFuzzer-MemcmpTest-DFSan -use_traces=1 -seed=1 -runs=2 -timeout=5 -verbosity=3 2>&1 | FileCheck %s -check-prefix=CHECK_DFSanMemcmpCallback
@@ -19,6 +14,3 @@ RUN: LLVMFuzzer-StrncmpTest-DFSan -use_t
RUN: not LLVMFuzzer-StrcmpTest-DFSan -use_traces=1 -seed=1 -runs=10000 -timeout=5 2>&1 | FileCheck %s --check-prefix=CHECK3
RUN: LLVMFuzzer-StrcmpTest-DFSan -use_traces=1 -seed=1 -runs=2 -timeout=5 -verbosity=3 2>&1 | FileCheck %s -check-prefix=CHECK_DFSanMemcmpCallback
-
-RUN: not LLVMFuzzer-SwitchTest-DFSan -use_traces=1 -seed=1 -runs=100000 -timeout=5 2>&1 | FileCheck %s --check-prefix=CHECK4
-RUN: LLVMFuzzer-SwitchTest-DFSan -use_traces=1 -seed=1 -runs=2 -timeout=5 -verbosity=3 2>&1 | FileCheck %s -check-prefix=CHECK_DFSanSwitchCallback
Removed: llvm/trunk/lib/Fuzzer/test/fuzzer-traces.test
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Fuzzer/test/fuzzer-traces.test?rev=281006&view=auto
==============================================================================
--- llvm/trunk/lib/Fuzzer/test/fuzzer-traces.test (original)
+++ llvm/trunk/lib/Fuzzer/test/fuzzer-traces.test (removed)
@@ -1,10 +0,0 @@
-CHECK: BINGO
-Done1000000: Done 1000000 runs in
-Done2000000: Done 2000000 runs in
-RUN: not LLVMFuzzer-SimpleCmpTest -use_traces=1 -seed=1 -runs=10000001 2>&1 | FileCheck %s
-
-RUN: not LLVMFuzzer-SwitchTest -use_traces=1 -seed=6 -runs=1000002 2>&1 | FileCheck %s
-RUN: LLVMFuzzer-SwitchTest -seed=7 -runs=1000000 2>&1 | FileCheck %s --check-prefix=Done1000000
-
-RUN: not LLVMFuzzer-SimpleHashTest -use_traces=1 -seed=8 -runs=2000000 -max_len=16 2>&1 | FileCheck %s
-RUN: LLVMFuzzer-SimpleHashTest -seed=9 -runs=2000000 -max_len=16 2>&1 | FileCheck %s --check-prefix=Done2000000
More information about the llvm-commits
mailing list