[compiler-rt] r354092 - [libFuzzer] print new functions as they are discovered in the fork mode

Tue Feb 19 11:15:51 PST 2019

On Tue, Feb 19, 2019 at 5:44 AM Peter Smith <peter.smith at linaro.org> wrote:

> Hello Kostya, Diana,
>
> I've taken over from Diana at looking after the AArch64 bots this
> week. I've run the cxa_atexit_race.cc test in a loop and it
> intermittently hangs on average about 1 in 75 times,

@Vitaly Buka <vitalybuka at google.com> who added the test.

> a frequency that
> seems to go up a lot when the machine is heavily loaded. I've tested
> before and after locally reverting r354092 I was able to reproduce a
> hang without r354092 so in conjunction with hanging test being in
> msan, I agree that it isn't this particular change that is the root
> cause.
>
> Given that the cxa_atexit_race.cc is relatively new (14th Februrary) I
> think it may just be that the fix for the atexit problem doesn't work
> on AArch64. I'll follow up with the author of the test to see if we
> can work something out.
>
> I don't know who maintains lsan on AArch64 either. I can ask our local
> expert on Thursday to see if he knows.

yes, please!

> Having said that while 18
> seconds is slow it is unlikely to cause the builder to run out of
> time.
>

the check-fuzzer test rule (and a few others) runs hundreds of tiny tests
that run lsan in the end.
On x86_64 it takes a fraction of a second, but on aarch64 it takes forver,
so it might very well be reason for other timeouts.

>
> Peter
>
> On Sat, 16 Feb 2019 at 02:06, Kostya Serebryany via llvm-commits
> <llvm-commits at lists.llvm.org> wrote:
> >
> > My change just triggered an older problem: lsan seems to be badly broken
> on aarch64.
> > It spends ridiculous amount of time, and a test that is expected to pass
> in < 1 second takes 18 seconds.
> >
> > % clang -fsanitize=leak
> ~/llvm/compiler-rt/test/lsan/TestCases/sanity_check_pure_c.c  && time
> ./a.out
> >
> > real    0m18.313s
> > user    0m18.304s
> > sys     0m0.008s
> >
> > Perhaps this is https://github.com/google/sanitizers/issues/703, or
> something else.
> >
> > My team doesn't maintain lsan on aarch64.. :(
> > I wonder who does?
> >
> >
> >
> >
> > On Fri, Feb 15, 2019 at 10:50 AM Kostya Serebryany <kcc at google.com>
> wrote:
> >>
> >> Hi Diana,
> >>
> >> I am not sure how to interpret these bot failures...
> >> Can you tell which test times out?
> >>
> >> The only tests that can be affected by this change pass:
> >>
> >> PASS: libFuzzer :: merge.test (48757 of 48763)
> >>
> >> PASS: libFuzzer :: fork.test (48754 of 48763)
> >>
> >> (let me revive my aarch64 development box and see it myself)
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Feb 15, 2019 at 4:22 AM Diana Picus <diana.picus at linaro.org>
> wrote:
> >>>
> >>> Hi Kostya,
> >>>
> >>> It seems that this is causing the tests to hang on AArch64:
> >>> http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/6571
> >>> http://lab.llvm.org:8011/builders/clang-cmake-aarch64-lld/builds/6038
> >>>
> >>> It might have something to do with cxa_atexit_race, since I saw it
> >>> running for about 8 minutes on one of the machines.
> >>>
> >>> Could you please have a look?
> >>>
> >>> Thanks,
> >>> Diana
> >>>
> >>> On Fri, 15 Feb 2019 at 02:21, Kostya Serebryany via llvm-commits
> >>> <llvm-commits at lists.llvm.org> wrote:
> >>> >
> >>> > Author: kcc
> >>> > Date: Thu Feb 14 17:22:00 2019
> >>> > New Revision: 354092
> >>> >
> >>> > URL: http://llvm.org/viewvc/llvm-project?rev=354092&view=rev
> >>> > Log:
> >>> > [libFuzzer] print new functions as they are discovered in the fork
> mode
> >>> >
> >>> > Modified:
> >>> >     compiler-rt/trunk/lib/fuzzer/FuzzerFork.cpp
> >>> >     compiler-rt/trunk/lib/fuzzer/FuzzerMerge.cpp
> >>> >     compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.cpp
> >>> >     compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.h
> >>> >
> >>> > Modified: compiler-rt/trunk/lib/fuzzer/FuzzerFork.cpp
> >>> > URL:
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerFork.cpp?rev=354092&r1=354091&r2=354092&view=diff
> >>> >
> ==============================================================================
> >>> > --- compiler-rt/trunk/lib/fuzzer/FuzzerFork.cpp (original)
> >>> > +++ compiler-rt/trunk/lib/fuzzer/FuzzerFork.cpp Thu Feb 14 17:22:00
> 2019
> >>> > @@ -13,6 +13,7 @@
> >>> >  #include "FuzzerIO.h"
> >>> >  #include "FuzzerMerge.h"
> >>> >  #include "FuzzerSHA1.h"
> >>> > +#include "FuzzerTracePC.h"
> >>> >  #include "FuzzerUtil.h"
> >>> >
> >>> >  #include <atomic>
> >>> > @@ -86,11 +87,13 @@ struct GlobalEnv {
> >>> >        Cmd.removeArgument(C);
> >>> >      Cmd.addFlag("reload", "0");  // working in an isolated dir, no
> reload.
> >>> >      Cmd.addFlag("print_final_stats", "1");
> >>> > +    Cmd.addFlag("print_funcs", "0");  // no need to spend time
> symbolizing.
> >>> >      Cmd.addFlag("max_total_time",
> std::to_string(std::min((size_t)300, JobId)));
> >>> >
> >>> >      auto Job = new FuzzJob;
> >>> >      std::string Seeds;
> >>> > -    if (size_t CorpusSubsetSize = std::min(Files.size(),
> (size_t)100))
> >>> > +    if (size_t CorpusSubsetSize =
> >>> > +            std::min(Files.size(), (size_t)sqrt(Files.size() + 2)))
> >>> >        for (size_t i = 0; i < CorpusSubsetSize; i++)
> >>> >          Seeds += (Seeds.empty() ? "" : ",") +
> >>> >                   Files[Rand->SkewTowardsLast(Files.size())];
> >>> > @@ -135,6 +138,12 @@ struct GlobalEnv {
> >>> >      RmDirRecursive(Job->CorpusDir);
> >>> >      Features.insert(NewFeatures.begin(), NewFeatures.end());
> >>> >      Cov.insert(NewCov.begin(), NewCov.end());
> >>> > +    for (auto Idx : NewCov)
> >>> > +      if (auto *TE = TPC.PCTableEntryByIdx(Idx))
> >>> > +        if (TPC.PcIsFuncEntry(TE))
> >>> > +          PrintPC("  NEW_FUNC: %p %F %L\n", "",
> >>> > +                  TPC.GetNextInstructionPc(TE->PC));
> >>> > +
> >>> >      auto Stats = ParseFinalStatsFromLog(Job->LogPath);
> >>> >      NumRuns += Stats.number_of_executed_units;
> >>> >      if (!FilesToAdd.empty())
> >>> >
> >>> > Modified: compiler-rt/trunk/lib/fuzzer/FuzzerMerge.cpp
> >>> > URL:
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerMerge.cpp?rev=354092&r1=354091&r2=354092&view=diff
> >>> >
> ==============================================================================
> >>> > --- compiler-rt/trunk/lib/fuzzer/FuzzerMerge.cpp (original)
> >>> > +++ compiler-rt/trunk/lib/fuzzer/FuzzerMerge.cpp Thu Feb 14 17:22:00
> 2019
> >>> > @@ -100,7 +100,7 @@ bool Merger::Parse(std::istream &IS, boo
> >>> >        LastSeenStartMarker = kInvalidStartMarker;
> >>> >        if (ParseCoverage) {
> >>> >          TmpFeatures.clear();  // use a vector from outer scope to
> avoid resizes.
> >>> > -        while (ISS1 >> std::hex >> N)
> >>> > +        while (ISS1 >> N)
> >>> >            TmpFeatures.push_back(N);
> >>> >          std::sort(TmpFeatures.begin(), TmpFeatures.end());
> >>> >          Files[CurrentFileIdx].Features = TmpFeatures;
> >>> > @@ -108,7 +108,7 @@ bool Merger::Parse(std::istream &IS, boo
> >>> >      } else if (Marker == "COV") {
> >>> >        size_t CurrentFileIdx = N;
> >>> >        if (ParseCoverage)
> >>> > -        while (ISS1 >> std::hex >> N)
> >>> > +        while (ISS1 >> N)
> >>> >            if (PCs.insert(N).second)
> >>> >              Files[CurrentFileIdx].Cov.push_back(N);
> >>> >      } else {
> >>> > @@ -220,7 +220,7 @@ void Fuzzer::CrashResistantMergeInternal
> >>> >      }
> >>> >      std::ostringstream StartedLine;
> >>> >      // Write the pre-run marker.
> >>> > -    OF << "STARTED " << std::dec << i << " " << U.size() << "\n";
> >>> > +    OF << "STARTED " << i << " " << U.size() << "\n";
> >>> >      OF.flush();  // Flush is important since Command::Execute may
> crash.
> >>> >      // Run.
> >>> >      TPC.ResetMaps();
> >>> > @@ -242,9 +242,9 @@ void Fuzzer::CrashResistantMergeInternal
> >>> >      // Write the post-run marker and the coverage.
> >>> >      OF << "FT " << i;
> >>> >      for (size_t F : UniqFeatures)
> >>> > -      OF << " " << std::hex << F;
> >>> > +      OF << " " << F;
> >>> >      OF << "\n";
> >>> > -    OF << "COV " << std::dec << i;
> >>> > +    OF << "COV " << i;
> >>> >      TPC.ForEachObservedPC([&](const TracePC::PCTableEntry *TE) {
> >>> >        if (AllPCs.insert(TE).second)
> >>> >          OF << " " << TPC.PCTableEntryIdx(TE);
> >>> >
> >>> > Modified: compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.cpp
> >>> > URL:
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.cpp?rev=354092&r1=354091&r2=354092&view=diff
> >>> >
> ==============================================================================
> >>> > --- compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.cpp (original)
> >>> > +++ compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.cpp Thu Feb 14
> 17:22:00 2019
> >>> > @@ -174,7 +174,7 @@ inline ALWAYS_INLINE uintptr_t GetPrevio
> >>> >
> >>> >  /// \return the address of the next instruction.
> >>> >  /// Note: the logic is copied from
> `sanitizer_common/sanitizer_stacktrace.cc`
> >>> > -inline ALWAYS_INLINE uintptr_t GetNextInstructionPc(uintptr_t PC) {
> >>> > +ALWAYS_INLINE uintptr_t TracePC::GetNextInstructionPc(uintptr_t PC)
> {
> >>> >  #if defined(__mips__)
> >>> >    return PC + 8;
> >>> >  #elif defined(__powerpc__) || defined(__sparc__) ||
> defined(__arm__) || \
> >>> > @@ -196,7 +196,7 @@ void TracePC::UpdateObservedPCs() {
> >>> >    };
> >>> >
> >>> >    auto Observe = [&](const PCTableEntry *TE) {
> >>> > -    if (TE->PCFlags & 1)
> >>> > +    if (PcIsFuncEntry(TE))
> >>> >        if (++ObservedFuncs[TE->PC] == 1 && NumPrintNewFuncs)
> >>> >          CoveredFuncs.push_back(TE->PC);
> >>> >      ObservePC(TE);
> >>> > @@ -239,6 +239,16 @@ uintptr_t TracePC::PCTableEntryIdx(const
> >>> >    return 0;
> >>> >  }
> >>> >
> >>> > +const TracePC::PCTableEntry *TracePC::PCTableEntryByIdx(uintptr_t
> Idx) {
> >>> > +  for (size_t i = 0; i < NumPCTables; i++) {
> >>> > +    auto &M = ModulePCTable[i];
> >>> > +    size_t Size = M.Stop - M.Start;
> >>> > +    if (Idx < Size) return &M.Start[Idx];
> >>> > +    Idx -= Size;
> >>> > +  }
> >>> > +  return nullptr;
> >>> > +}
> >>> > +
> >>> >  static std::string GetModuleName(uintptr_t PC) {
> >>> >    char ModulePathRaw[4096] = "";  // What's PATH_MAX in portable
> C++?
> >>> >    void *OffsetRaw = nullptr;
> >>> > @@ -257,10 +267,10 @@ void TracePC::IterateCoveredFunctions(Ca
> >>> >      auto ModuleName = GetModuleName(M.Start->PC);
> >>> >      for (auto NextFE = M.Start; NextFE < M.Stop; ) {
> >>> >        auto FE = NextFE;
> >>> > -      assert((FE->PCFlags & 1) && "Not a function entry point");
> >>> > +      assert(PcIsFuncEntry(FE) && "Not a function entry point");
> >>> >        do {
> >>> >          NextFE++;
> >>> > -      } while (NextFE < M.Stop && !(NextFE->PCFlags & 1));
> >>> > +      } while (NextFE < M.Stop && !(PcIsFuncEntry(NextFE)));
> >>> >        CB(FE, NextFE, ObservedFuncs[FE->PC]);
> >>> >      }
> >>> >    }
> >>> > @@ -275,7 +285,7 @@ void TracePC::SetFocusFunction(const std
> >>> >      auto &PCTE = ModulePCTable[M];
> >>> >      size_t N = PCTE.Stop - PCTE.Start;
> >>> >      for (size_t I = 0; I < N; I++) {
> >>> > -      if (!(PCTE.Start[I].PCFlags & 1)) continue;  // not a
> function entry.
> >>> > +      if (!(PcIsFuncEntry(&PCTE.Start[I]))) continue;  // not a
> function entry.
> >>> >        auto Name = DescribePC("%F",
> GetNextInstructionPc(PCTE.Start[I].PC));
> >>> >        if (Name[0] == 'i' && Name[1] == 'n' && Name[2] == ' ')
> >>> >          Name = Name.substr(3, std::string::npos);
> >>> >
> >>> > Modified: compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.h
> >>> > URL:
> http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.h?rev=354092&r1=354091&r2=354092&view=diff
> >>> >
> ==============================================================================
> >>> > --- compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.h (original)
> >>> > +++ compiler-rt/trunk/lib/fuzzer/FuzzerTracePC.h Thu Feb 14 17:22:00
> 2019
> >>> > @@ -127,6 +127,9 @@ class TracePC {
> >>> >    };
> >>> >
> >>> >    uintptr_t PCTableEntryIdx(const PCTableEntry *TE);
> >>> > +  const PCTableEntry *PCTableEntryByIdx(uintptr_t Idx);
> >>> > +  static uintptr_t GetNextInstructionPc(uintptr_t PC);
> >>> > +  bool PcIsFuncEntry(const PCTableEntry *TE) { return TE->PCFlags &
> 1; }
> >>> >
> >>> >  private:
> >>> >    bool UseCounters = false;
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > llvm-commits mailing list
> >>> > llvm-commits at lists.llvm.org
> >>> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> >
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190219/a3260761/attachment.html>