[PATCH] D75390: Fix GSYM tests to run the yaml files and fix test failures on some machines.

Wed Mar 4 10:05:58 PST 2020

hctim added a comment.

This patch broke <http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/39174> the MSan buildbot. The MSan bot is fickle to work (as you need a sanitized libcxx), the easiest way to reproduce the issue is following these <https://github.com/google/sanitizers/wiki/SanitizerBotReproduceBuild> instructions.

Full MSan report with `-fsanitize-memory-track-origins=2`:

  Uninitialized bytes in __interceptor_write at offset 24 inside [0x720000002000, 118)
  ==146908==WARNING: MemorySanitizer: use-of-uninitialized-value
      #0 0x5d94bc7 in llvm::raw_fd_ostream::write_impl(char const*, unsigned long) /usr/local/google/home/mitchp/llvm/llvm/lib/Support/raw_ostream.cpp:712:19
      #1 0x5d94fc2 in flush_nonempty /usr/local/google/home/mitchp/llvm/llvm/lib/Support/raw_ostream.cpp:219:3
      #2 0x5d94fc2 in flush /usr/local/google/home/mitchp/llvm/llvm/include/llvm/Support/raw_ostream.h:157:7
      #3 0x5d94fc2 in llvm::raw_fd_ostream::seek(unsigned long) /usr/local/google/home/mitchp/llvm/llvm/lib/Support/raw_ostream.cpp:754:3
      #4 0x5d95493 in llvm::raw_fd_ostream::pwrite_impl(char const*, unsigned long, unsigned long) /usr/local/google/home/mitchp/llvm/llvm/lib/Support/raw_ostream.cpp:770:3
      #5 0x567ee45 in pwrite /usr/local/google/home/mitchp/llvm/llvm/include/llvm/Support/raw_ostream.h:387:5
      #6 0x567ee45 in llvm::gsym::FileWriter::fixup32(unsigned int, unsigned long) /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/FileWriter.cpp:55:6
      #7 0x56847a4 in llvm::gsym::FunctionInfo::encode(llvm::gsym::FileWriter&) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/FunctionInfo.cpp:126:7
      #8 0x566321a in llvm::gsym::GsymCreator::encode(llvm::gsym::FileWriter&) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/GsymCreator.cpp:146:51
      #9 0x56619f9 in llvm::gsym::GsymCreator::save(llvm::StringRef, llvm::support::endianness) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/GsymCreator.cpp:56:10
      #10 0x1736921 in handleObjectFile(llvm::object::ObjectFile&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:335:23
      #11 0x1726afa in handleBuffer /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:391:22
      #12 0x1726afa in handleFileConversionToGSYM /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:404:10
      #13 0x1726afa in convertFileToGSYM /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:422:20
      #14 0x1726afa in main /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:463:20
      #15 0x7f483a67bbba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba)
      #16 0x16ac059 in _start (/usr/local/google/home/mitchp/llvm-build/msan/sanitized-clang/bin/llvm-gsymutil+0x16ac059)

    Uninitialized value was stored to memory at
      #0 0x5d8eb94 in copy_to_buffer /usr/local/google/home/mitchp/llvm/llvm/lib/Support/raw_ostream.cpp:294:24
      #1 0x5d8eb94 in llvm::raw_ostream::write(char const*, unsigned long) /usr/local/google/home/mitchp/llvm/llvm/lib/Support/raw_ostream.cpp:280:3
      #2 0x567ea18 in llvm::gsym::FileWriter::writeU32(unsigned int) /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/FileWriter.cpp:45:6
      #3 0x568dd27 in llvm::gsym::Header::encode(llvm::gsym::FileWriter&) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/Header.cpp:96:5
      #4 0x5662275 in llvm::gsym::GsymCreator::encode(llvm::gsym::FileWriter&) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/GsymCreator.cpp:101:25
      #5 0x56619f9 in llvm::gsym::GsymCreator::save(llvm::StringRef, llvm::support::endianness) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/GsymCreator.cpp:56:10
      #6 0x1736921 in handleObjectFile(llvm::object::ObjectFile&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:335:23
      #7 0x1726afa in handleBuffer /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:391:22
      #8 0x1726afa in handleFileConversionToGSYM /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:404:10
      #9 0x1726afa in convertFileToGSYM /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:422:20
      #10 0x1726afa in main /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:463:20
      #11 0x7f483a67bbba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba)

    Uninitialized value was stored to memory at
      #0 0x567ea3a in llvm::gsym::FileWriter::writeU32(unsigned int) /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/FileWriter.cpp:44:18
      #1 0x568dd27 in llvm::gsym::Header::encode(llvm::gsym::FileWriter&) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/Header.cpp:96:5
      #2 0x5662275 in llvm::gsym::GsymCreator::encode(llvm::gsym::FileWriter&) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/GsymCreator.cpp:101:25
      #3 0x56619f9 in llvm::gsym::GsymCreator::save(llvm::StringRef, llvm::support::endianness) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/GsymCreator.cpp:56:10
      #4 0x1736921 in handleObjectFile(llvm::object::ObjectFile&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:335:23
      #5 0x1726afa in handleBuffer /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:391:22
      #6 0x1726afa in handleFileConversionToGSYM /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:404:10
      #7 0x1726afa in convertFileToGSYM /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:422:20
      #8 0x1726afa in main /usr/local/google/home/mitchp/llvm/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp:463:20
      #9 0x7f483a67bbba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba)

    Uninitialized value was created by an allocation of 'Hdr' in the stack frame of function '_ZNK4llvm4gsym11GsymCreator6encodeERNS0_10FileWriterE'
      #0 0x5661a80 in llvm::gsym::GsymCreator::encode(llvm::gsym::FileWriter&) const /usr/local/google/home/mitchp/llvm/llvm/lib/DebugInfo/GSYM/GsymCreator.cpp:59

  SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/local/google/home/mitchp/llvm/llvm/lib/Support/raw_ostream.cpp:712:19 in llvm::raw_fd_ostream::write_impl(char const*, unsigned long)
  Exiting

>From the bot:

  -- Testing: 36342 tests, 96 workers --
  Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
  FAIL: LLVM :: tools/llvm-gsymutil/X86/elf-dwarf.yaml (35798 of 36342)
  ******************** TEST 'LLVM :: tools/llvm-gsymutil/X86/elf-dwarf.yaml' FAILED ********************
  Script:
  --
  : 'RUN: at line 5';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/yaml2obj /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/elf-dwarf.yaml -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/elf-dwarf.yaml.tmp
  : 'RUN: at line 6';   llvm-gsymutil --convert /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/elf-dwarf.yaml.tmp -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/elf-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/elf-dwarf.yaml --check-prefix=CONVERT
  : 'RUN: at line 7';   llvm-gsymutil --address=0x400391 --address=0x4004cd /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/elf-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/elf-dwarf.yaml --check-prefix=ADDR
  : 'RUN: at line 8';   llvm-gsymutil --address=0x400391 --address=0x4004cd --verbose /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/elf-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/elf-dwarf.yaml --check-prefix=ADDRV --dump-input=always
  : 'RUN: at line 9';   llvm-gsymutil /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/elf-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/elf-dwarf.yaml --check-prefix=DUMP
  --
  Exit Code: 1

  Command Output (stderr):
  --
  /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/elf-dwarf.yaml:29:12: error: CONVERT: expected string not found in input
  # CONVERT: Input file: {{.*\.yaml\.tmp}}
             ^
  <stdin>:1:1: note: scanning from here
  Uninitialized bytes in __interceptor_write at offset 24 inside [0x720000003000, 395)
  ^
  <stdin>:18:61: note: possible intended match here
   #15 0x7fcccffc82e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
                                                              ^

  --

  ********************
  Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
  FAIL: LLVM :: tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml (35814 of 36342)
  ******************** TEST 'LLVM :: tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml' FAILED ********************
  Script:
  --
  : 'RUN: at line 9';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/yaml2obj /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/ARM_AArch64/Output/fat-macho-dwarf.yaml.tmp
  : 'RUN: at line 10';   llvm-gsymutil --convert /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/ARM_AArch64/Output/fat-macho-dwarf.yaml.tmp --arch armv7 -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/ARM_AArch64/Output/fat-macho-dwarf.yaml.tmp.armv7.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml --check-prefix=ARMV7
  : 'RUN: at line 11';   llvm-gsymutil --convert /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/ARM_AArch64/Output/fat-macho-dwarf.yaml.tmp --arch arm64 -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/ARM_AArch64/Output/fat-macho-dwarf.yaml.tmp.arm64.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml --check-prefix=ARM64
  : 'RUN: at line 12';   llvm-gsymutil --convert /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/ARM_AArch64/Output/fat-macho-dwarf.yaml.tmp -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/ARM_AArch64/Output/fat-macho-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml --check-prefix=FAT
  --
  Exit Code: 1

  Command Output (stderr):
  --
  /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml:14:10: error: ARMV7: expected string not found in input
  # ARMV7: Input file: {{.*\.yaml\.tmp}}
           ^
  <stdin>:1:1: note: scanning from here
  Uninitialized bytes in __interceptor_write at offset 24 inside [0x720000002000, 118)
  ^
  <stdin>:18:61: note: possible intended match here
   #15 0x7f561e2c92e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
                                                              ^

  --

  ********************
  Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
  FAIL: LLVM :: tools/llvm-gsymutil/X86/mach-dwarf.yaml (35815 of 36342)
  ******************** TEST 'LLVM :: tools/llvm-gsymutil/X86/mach-dwarf.yaml' FAILED ********************
  Script:
  --
  : 'RUN: at line 5';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/yaml2obj /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/mach-dwarf.yaml -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/mach-dwarf.yaml.tmp
  : 'RUN: at line 6';   llvm-gsymutil --convert /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/mach-dwarf.yaml.tmp -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/mach-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/mach-dwarf.yaml --check-prefix=CONVERT
  : 'RUN: at line 7';   llvm-gsymutil --address=0 --address=0x100000000 --address=0x100000f90 --address=0x100000faa --address=0x200000000 /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/mach-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/mach-dwarf.yaml --check-prefix=ADDR
  : 'RUN: at line 8';   llvm-gsymutil --verbose --address=0x100000000 --address=0x100000f90 --address=0x100000faa /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/mach-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/mach-dwarf.yaml --check-prefix=ADDRV
  : 'RUN: at line 9';   llvm-gsymutil /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/test/tools/llvm-gsymutil/X86/Output/mach-dwarf.yaml.tmp.gsym 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/mach-dwarf.yaml --check-prefix=DUMP
  --
  Exit Code: 1

  Command Output (stderr):
  --
  /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/tools/llvm-gsymutil/X86/mach-dwarf.yaml:11:12: error: CONVERT: expected string not found in input
  # CONVERT: Input file: {{.*\.yaml\.tmp}}
             ^
  <stdin>:1:1: note: scanning from here
  Uninitialized bytes in __interceptor_write at offset 24 inside [0x720000003000, 130)
  ^
  <stdin>:18:61: note: possible intended match here
   #15 0x7fc3869db2e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
                                                              ^

  --

  ********************
  Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 

  Testing Time: 124.12s
  ********************
  Failing Tests (3):
      LLVM :: tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml
      LLVM :: tools/llvm-gsymutil/X86/elf-dwarf.yaml
      LLVM :: tools/llvm-gsymutil/X86/mach-dwarf.yaml

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75390/new/

https://reviews.llvm.org/D75390