[llvm] a3b0dde - Reland: [llvm-cov] Look up object files using debuginfod
Daniel Thornburgh via llvm-commits
llvm-commits at lists.llvm.org
Fri Jan 27 12:40:42 PST 2023
Thanks, I sent a fix-forward for this. There was also a logic error with
the comparator functions that this uncovered; unique takes ==, while sort
and set_difference take <.
On Fri, Jan 27, 2023 at 6:37 AM Aaron Ballman <aaron at aaronballman.com>
wrote:
> This appears to be causing new diagnostics in MSVC that should be
> addressed:
>
> Warning C4858 discarding return value: The 'unique' algorithm returns
> the iterator past the last element that should be kept. You need to
> call container.erase(result, container.end()) afterwards.
> F:\source\llvm-project\llvm\out\build\x64-Debug\llvm
> F:\source\llvm-project\llvm\lib\ProfileData\Coverage\CoverageMapping.cpp
> 421
> Warning C4858 discarding return value: The 'unique' algorithm returns
> the iterator past the last element that should be kept. You need to
> call container.erase(result, container.end()) afterwards.
> F:\source\llvm-project\llvm\out\build\x64-Debug\llvm
> F:\source\llvm-project\llvm\lib\ProfileData\Coverage\CoverageMapping.cpp
> 426
>
> ~Aaron
>
> On Thu, Jan 26, 2023 at 3:59 PM Daniel Thornburgh via llvm-commits
> <llvm-commits at lists.llvm.org> wrote:
> >
> >
> > Author: Daniel Thornburgh
> > Date: 2023-01-26T12:59:52-08:00
> > New Revision: a3b0dde4edb93248f74d86c63b0ae108854004e6
> >
> > URL:
> https://github.com/llvm/llvm-project/commit/a3b0dde4edb93248f74d86c63b0ae108854004e6
> > DIFF:
> https://github.com/llvm/llvm-project/commit/a3b0dde4edb93248f74d86c63b0ae108854004e6.diff
> >
> > LOG: Reland: [llvm-cov] Look up object files using debuginfod
> >
> > Reviewed By: gulfem
> >
> > Differential Revision: https://reviews.llvm.org/D136702
> >
> > Added:
> > compiler-rt/test/profile/Linux/binary-id-debuginfod.c
> > compiler-rt/test/profile/Linux/binary-id-lookup.c
> >
> > Modified:
> > compiler-rt/test/profile/CMakeLists.txt
> > compiler-rt/test/profile/Linux/lit.local.cfg.py
> > compiler-rt/test/profile/lit.site.cfg.py.in
> > llvm/cmake/modules/LLVMConfig.cmake.in
> > llvm/docs/CommandGuide/llvm-cov.rst
> > llvm/include/llvm/Debuginfod/Debuginfod.h
> > llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
> > llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h
> > llvm/lib/Debuginfod/Debuginfod.cpp
> > llvm/lib/ProfileData/Coverage/CoverageMapping.cpp
> > llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp
> > llvm/tools/llvm-cov/CMakeLists.txt
> > llvm/tools/llvm-cov/CodeCoverage.cpp
> > llvm/tools/llvm-objdump/llvm-objdump.cpp
> > llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp
> >
> > Removed:
> >
> >
> >
> >
> ################################################################################
> > diff --git a/compiler-rt/test/profile/CMakeLists.txt
> b/compiler-rt/test/profile/CMakeLists.txt
> > index 782be305a13ca..1b61d1ea0728e 100644
> > --- a/compiler-rt/test/profile/CMakeLists.txt
> > +++ b/compiler-rt/test/profile/CMakeLists.txt
> > @@ -16,6 +16,8 @@ if(APPLE)
> > darwin_filter_host_archs(PROFILE_SUPPORTED_ARCH PROFILE_TEST_ARCH)
> > endif()
> >
> > +pythonize_bool(LLVM_ENABLE_CURL)
> > +
> > foreach(arch ${PROFILE_TEST_ARCH})
> > set(PROFILE_TEST_TARGET_ARCH ${arch})
> > get_test_cc_for_arch(${arch} PROFILE_TEST_TARGET_CC
> PROFILE_TEST_TARGET_CFLAGS)
> >
> > diff --git a/compiler-rt/test/profile/Linux/binary-id-debuginfod.c
> b/compiler-rt/test/profile/Linux/binary-id-debuginfod.c
> > new file mode 100644
> > index 0000000000000..39a125b8c09f7
> > --- /dev/null
> > +++ b/compiler-rt/test/profile/Linux/binary-id-debuginfod.c
> > @@ -0,0 +1,33 @@
> > +// REQUIRES: linux, curl
> > +// RUN: split-file %s %t
> > +// RUN: %clang_profgen -Wl,--build-id=0x12345678 -fcoverage-mapping -O2
> -shared %t/foo.c -o %t/libfoo.so
> > +// RUN: %clang_profgen -Wl,--build-id=0xabcd1234 -fcoverage-mapping -O2
> %t/main.c -L%t -lfoo -o %t.main
> > +// RUN: rm -rf %t.profdir
> > +// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw
> LD_LIBRARY_PATH=%t %run %t.main
> > +// RUN: mkdir -p %t/buildid/12345678 %t/buildid/abcd1234
> > +// RUN: cp %t/libfoo.so %t/buildid/12345678/debuginfo
> > +// RUN: cp %t.main %t/buildid/abcd1234/debuginfo
> > +// RUN: llvm-profdata merge -o %t.profdata %t.profdir/default_*.profraw
> > +// RUN: env DEBUGINFOD_URLS=file://%t llvm-cov show -instr-profile
> %t.profdata | FileCheck %s
> > +// RUN: echo "bad" > %t/libfoo.so %t/buildid/12345678/debuginfo
> > +// RUN: echo "bad" > %t/buildid/abcd1234/debuginfo
> > +// RUN: env DEBUGINFOD_URLS=file://%t llvm-cov show -instr-profile
> %t.profdata -debuginfod=false %t.main | FileCheck %s
> --check-prefix=NODEBUGINFOD
> > +
> > +// CHECK: 1| 1|void foo(void) {}
> > +// CHECK: 2| 1|void bar(void) {}
> > +// CHECK: 3| 1|int main() {
> > +
> > +// NODEBUGINFOD-NOT: foo(void) {}
> > +// NODEBUGINFOD: main
> > +
> > +//--- foo.c
> > +void foo(void) {}
> > +
> > +//--- main.c
> > +void foo(void);
> > +void bar(void) {}
> > +int main() {
> > + foo();
> > + bar();
> > + return 0;
> > +}
> >
> > diff --git a/compiler-rt/test/profile/Linux/binary-id-lookup.c
> b/compiler-rt/test/profile/Linux/binary-id-lookup.c
> > new file mode 100644
> > index 0000000000000..9cc6648903e1d
> > --- /dev/null
> > +++ b/compiler-rt/test/profile/Linux/binary-id-lookup.c
> > @@ -0,0 +1,32 @@
> > +// REQUIRES: linux
> > +// RUN: split-file %s %t
> > +// RUN: %clang_profgen -Wl,--build-id=0x12345678 -fcoverage-mapping -O2
> -shared %t/foo.c -o %t/libfoo.so
> > +// RUN: %clang_profgen -Wl,--build-id=0xabcd1234 -fcoverage-mapping -O2
> %t/main.c -L%t -lfoo -o %t.main
> > +// RUN: rm -rf %t.profdir
> > +// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw
> LD_LIBRARY_PATH=%t %run %t.main
> > +// RUN: mkdir -p %t/.build-id/12 %t/.build-id/ab
> > +// RUN: cp %t/libfoo.so %t/.build-id/12/345678.debug
> > +// RUN: cp %t.main %t/.build-id/ab/cd1234.debug
> > +// RUN: llvm-profdata merge -o %t.profdata %t.profdir/default_*.profraw
> > +// RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory
> %t | FileCheck %s
> > +// RUN: echo "bad" > %t/.build-id/ab/cd1234.debug
> > +// RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory
> %t %t.main | FileCheck %s
> > +// RUN: not llvm-cov show -instr-profile %t.profdata
> -debug-file-directory %t/empty 2>&1 | FileCheck %s --check-prefix=NODATA
> > +
> > +// CHECK: 1| 1|void foo(void) {}
> > +// CHECK: 2| 1|void bar(void) {}
> > +// CHECK: 3| 1|int main() {
> > +
> > +// NODATA: error: Failed to load coverage: '': No coverage data found
> > +
> > +//--- foo.c
> > +void foo(void) {}
> > +
> > +//--- main.c
> > +void foo(void);
> > +void bar(void) {}
> > +int main() {
> > + foo();
> > + bar();
> > + return 0;
> > +}
> >
> > diff --git a/compiler-rt/test/profile/Linux/lit.local.cfg.py
> b/compiler-rt/test/profile/Linux/lit.local.cfg.py
> > index 98d79518b92aa..9bb92b7f14fe6 100644
> > --- a/compiler-rt/test/profile/Linux/lit.local.cfg.py
> > +++ b/compiler-rt/test/profile/Linux/lit.local.cfg.py
> > @@ -38,3 +38,6 @@ def is_gold_linker_available():
> >
> > if root.host_os not in ['Linux'] or not is_gold_linker_available():
> > config.unsupported = True
> > +
> > +if config.have_curl:
> > + config.available_features.add('curl')
> >
> > diff --git a/compiler-rt/test/profile/lit.site.cfg.py.in
> b/compiler-rt/test/profile/lit.site.cfg.py.in
> > index bb81fb0c93f75..3466eabf7248d 100644
> > --- a/compiler-rt/test/profile/lit.site.cfg.py.in
> > +++ b/compiler-rt/test/profile/lit.site.cfg.py.in
> > @@ -4,6 +4,7 @@
> > config.profile_lit_binary_dir = "@PROFILE_LIT_BINARY_DIR@"
> > config.target_cflags = "@PROFILE_TEST_TARGET_CFLAGS@"
> > config.target_arch = "@PROFILE_TEST_TARGET_ARCH@"
> > +config.have_curl = @LLVM_ENABLE_CURL_PYBOOL@
> >
> > # Load common config for all compiler-rt lit tests.
> > lit_config.load_config(config, "@COMPILER_RT_BINARY_DIR@
> /test/lit.common.configured")
> >
> > diff --git a/llvm/cmake/modules/LLVMConfig.cmake.in
> b/llvm/cmake/modules/LLVMConfig.cmake.in
> > index bae2a07c0fb16..2d90512bfb06e 100644
> > --- a/llvm/cmake/modules/LLVMConfig.cmake.in
> > +++ b/llvm/cmake/modules/LLVMConfig.cmake.in
> > @@ -83,6 +83,11 @@ if(LLVM_ENABLE_LIBXML2)
> > find_package(LibXml2)
> > endif()
> >
> > +set(LLVM_ENABLE_CURL @LLVM_ENABLE_CURL@)
> > +if(LLVM_ENABLE_CURL)
> > + find_package(CURL)
> > +endif()
> > +
> > set(LLVM_WITH_Z3 @LLVM_WITH_Z3@)
> >
> > set(LLVM_ENABLE_DIA_SDK @LLVM_ENABLE_DIA_SDK@)
> >
> > diff --git a/llvm/docs/CommandGuide/llvm-cov.rst
> b/llvm/docs/CommandGuide/llvm-cov.rst
> > index 319835de168a9..495faa7201e18 100644
> > --- a/llvm/docs/CommandGuide/llvm-cov.rst
> > +++ b/llvm/docs/CommandGuide/llvm-cov.rst
> > @@ -349,6 +349,17 @@ OPTIONS
> > coverage >= high, red when coverage < low, and yellow otherwise. Both
> high and
> > low should be between 0-100 and high > low.
> >
> > +.. option:: -debuginfod
> > +
> > +Use debuginfod to look up coverage mapping for binary IDs present in
> the profile
> > +but not in any object given on the command line. Defaults to true if
> debuginfod
> > +is compiled in and configured via the DEBUGINFOD_URLS environment
> variable.
> > +
> > +.. option:: -debug-file-directory=<dir>
> > +
> > +Provides local directories to search for objects corresponding to
> binary IDs in
> > +the profile (as with debuginfod). Defaults to system build ID
> directories.
> > +
> > .. program:: llvm-cov report
> >
> > .. _llvm-cov-report:
> > @@ -418,6 +429,18 @@ OPTIONS
> > when binaries have been compiled with one of `-fcoverage-prefix-map`
> > `-fcoverage-compilation-dir`, or `-ffile-compilation-dir`.
> >
> > +.. option:: -debuginfod
> > +
> > +Attempt to look up coverage mapping from objects using debuginfod. This
> is
> > +attempted by default for binary IDs present in the profile but not
> provided on
> > +the command line, so long as debuginfod is compiled in and configured
> via
> > +DEBUGINFOD_URLS.
> > +
> > +.. option:: -debug-file-directory=<dir>
> > +
> > +Provides a directory to search for objects corresponding to binary IDs
> in the
> > +profile.
> > +
> > .. program:: llvm-cov export
> >
> > .. _llvm-cov-export:
> > @@ -492,3 +515,15 @@ OPTIONS
> > Directory used as a base for relative coverage mapping paths. Only
> applicable
> > when binaries have been compiled with one of `-fcoverage-prefix-map`
> > `-fcoverage-compilation-dir`, or `-ffile-compilation-dir`.
> > +
> > +.. option:: -debuginfod
> > +
> > +Attempt to look up coverage mapping from objects using debuginfod. This
> is
> > +attempted by default for binary IDs present in the profile but not
> provided on
> > +the command line, so long as debuginfod is compiled in and configured
> via
> > +DEBUGINFOD_URLS.
> > +
> > +.. option:: -debug-file-directory=<dir>
> > +
> > +Provides a directory to search for objects corresponding to binary IDs
> in the
> > +profile.
> >
> > diff --git a/llvm/include/llvm/Debuginfod/Debuginfod.h
> b/llvm/include/llvm/Debuginfod/Debuginfod.h
> > index caece0e6fc194..ec7f5691dda4f 100644
> > --- a/llvm/include/llvm/Debuginfod/Debuginfod.h
> > +++ b/llvm/include/llvm/Debuginfod/Debuginfod.h
> > @@ -38,9 +38,13 @@
> >
> > namespace llvm {
> >
> > +/// Returns false if a debuginfod lookup can be determined to have no
> chance of
> > +/// succeeding.
> > +bool canUseDebuginfod();
> > +
> > /// Finds default array of Debuginfod server URLs by checking
> DEBUGINFOD_URLS
> > /// environment variable.
> > -Expected<SmallVector<StringRef>> getDefaultDebuginfodUrls();
> > +SmallVector<StringRef> getDefaultDebuginfodUrls();
> >
> > /// Finds a default local file caching directory for the debuginfod
> client,
> > /// first checking DEBUGINFOD_CACHE_PATH.
> >
> > diff --git a/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
> b/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
> > index 4d48308d5509e..bdb7728624686 100644
> > --- a/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
> > +++ b/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h
> > @@ -21,6 +21,7 @@
> > #include "llvm/ADT/StringRef.h"
> > #include "llvm/ADT/iterator.h"
> > #include "llvm/ADT/iterator_range.h"
> > +#include "llvm/Object/BuildID.h"
> > #include "llvm/ProfileData/InstrProf.h"
> > #include "llvm/Support/Alignment.h"
> > #include "llvm/Support/Compiler.h"
> > @@ -42,6 +43,10 @@ namespace llvm {
> >
> > class IndexedInstrProfReader;
> >
> > +namespace object {
> > +class BuildIDFetcher;
> > +} // namespace object
> > +
> > namespace coverage {
> >
> > class CoverageMappingReader;
> > @@ -579,6 +584,13 @@ class CoverageMapping {
> > ArrayRef<std::unique_ptr<CoverageMappingReader>> CoverageReaders,
> > IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage);
> >
> > + // Load coverage records from file.
> > + static Error
> > + loadFromFile(StringRef Filename, StringRef Arch, StringRef
> CompilationDir,
> > + IndexedInstrProfReader &ProfileReader, CoverageMapping
> &Coverage,
> > + bool &DataFound,
> > + SmallVectorImpl<object::BuildID> *FoundBinaryIDs =
> nullptr);
> > +
> > /// Add a function record corresponding to \p Record.
> > Error loadFunctionRecord(const CoverageMappingRecord &Record,
> > IndexedInstrProfReader &ProfileReader);
> > @@ -604,8 +616,8 @@ class CoverageMapping {
> > /// Ignores non-instrumented object files unless all are not
> instrumented.
> > static Expected<std::unique_ptr<CoverageMapping>>
> > load(ArrayRef<StringRef> ObjectFilenames, StringRef ProfileFilename,
> > - ArrayRef<StringRef> Arches = std::nullopt,
> > - StringRef CompilationDir = "");
> > + ArrayRef<StringRef> Arches = std::nullopt, StringRef
> CompilationDir = "",
> > + const object::BuildIDFetcher *BIDFetcher = nullptr);
> >
> > /// The number of functions that couldn't have their profiles mapped.
> > ///
> >
> > diff --git
> a/llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h
> b/llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h
> > index 39c0045369be7..326c1b0d33384 100644
> > --- a/llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h
> > +++ b/llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h
> > @@ -205,7 +205,8 @@ class BinaryCoverageReader : public
> CoverageMappingReader {
> > static Expected<std::vector<std::unique_ptr<BinaryCoverageReader>>>
> > create(MemoryBufferRef ObjectBuffer, StringRef Arch,
> > SmallVectorImpl<std::unique_ptr<MemoryBuffer>>
> &ObjectFileBuffers,
> > - StringRef CompilationDir = "");
> > + StringRef CompilationDir = "",
> > + SmallVectorImpl<object::BuildIDRef> *BinaryIDs = nullptr);
> >
> > static Expected<std::unique_ptr<BinaryCoverageReader>>
> > createCoverageReaderFromBuffer(StringRef Coverage,
> >
> > diff --git a/llvm/lib/Debuginfod/Debuginfod.cpp
> b/llvm/lib/Debuginfod/Debuginfod.cpp
> > index 026f118bbf5b5..2b0710b536bae 100644
> > --- a/llvm/lib/Debuginfod/Debuginfod.cpp
> > +++ b/llvm/lib/Debuginfod/Debuginfod.cpp
> > @@ -55,7 +55,11 @@ static std::string buildIDToString(BuildIDRef ID) {
> > return llvm::toHex(ID, /*LowerCase=*/true);
> > }
> >
> > -Expected<SmallVector<StringRef>> getDefaultDebuginfodUrls() {
> > +bool canUseDebuginfod() {
> > + return HTTPClient::isAvailable() &&
> !getDefaultDebuginfodUrls().empty();
> > +}
> > +
> > +SmallVector<StringRef> getDefaultDebuginfodUrls() {
> > const char *DebuginfodUrlsEnv = std::getenv("DEBUGINFOD_URLS");
> > if (DebuginfodUrlsEnv == nullptr)
> > return SmallVector<StringRef>();
> > @@ -126,13 +130,8 @@ Expected<std::string>
> getCachedOrDownloadArtifact(StringRef UniqueKey,
> > return CacheDirOrErr.takeError();
> > CacheDir = *CacheDirOrErr;
> >
> > - Expected<SmallVector<StringRef>> DebuginfodUrlsOrErr =
> > - getDefaultDebuginfodUrls();
> > - if (!DebuginfodUrlsOrErr)
> > - return DebuginfodUrlsOrErr.takeError();
> > - SmallVector<StringRef> &DebuginfodUrls = *DebuginfodUrlsOrErr;
> > return getCachedOrDownloadArtifact(UniqueKey, UrlPath, CacheDir,
> > - DebuginfodUrls,
> > + getDefaultDebuginfodUrls(),
> > getDefaultDebuginfodTimeout());
> > }
> >
> > @@ -159,7 +158,8 @@ class StreamedHTTPResponseHandler : public
> HTTPResponseHandler {
> >
> > Error StreamedHTTPResponseHandler::handleBodyChunk(StringRef BodyChunk)
> {
> > if (!FileStream) {
> > - if (Client.responseCode() != 200)
> > + unsigned Code = Client.responseCode();
> > + if (Code && Code != 200)
> > return Error::success();
> > Expected<std::unique_ptr<CachedFileStream>> FileStreamOrError =
> > CreateStream();
> > @@ -259,7 +259,8 @@ Expected<std::string> getCachedOrDownloadArtifact(
> > if (Err)
> > return std::move(Err);
> >
> > - if (Client.responseCode() != 200)
> > + unsigned Code = Client.responseCode();
> > + if (Code && Code != 200)
> > continue;
> >
> > // Return the path to the artifact on disk.
> >
> > diff --git a/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp
> b/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp
> > index 6113f78aeb4ee..9e3a3d7ba40fc 100644
> > --- a/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp
> > +++ b/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp
> > @@ -17,6 +17,7 @@
> > #include "llvm/ADT/SmallBitVector.h"
> > #include "llvm/ADT/SmallVector.h"
> > #include "llvm/ADT/StringRef.h"
> > +#include "llvm/Object/BuildID.h"
> > #include "llvm/ProfileData/Coverage/CoverageMappingReader.h"
> > #include "llvm/ProfileData/InstrProfReader.h"
> > #include "llvm/Support/Debug.h"
> > @@ -342,10 +343,49 @@ static Error handleMaybeNoDataFoundError(Error E) {
> > });
> > }
> >
> > +Error CoverageMapping::loadFromFile(
> > + StringRef Filename, StringRef Arch, StringRef CompilationDir,
> > + IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage,
> > + bool &DataFound, SmallVectorImpl<object::BuildID> *FoundBinaryIDs) {
> > + auto CovMappingBufOrErr = MemoryBuffer::getFileOrSTDIN(
> > + Filename, /*IsText=*/false, /*RequiresNullTerminator=*/false);
> > + if (std::error_code EC = CovMappingBufOrErr.getError())
> > + return createFileError(Filename, errorCodeToError(EC));
> > + MemoryBufferRef CovMappingBufRef =
> > + CovMappingBufOrErr.get()->getMemBufferRef();
> > + SmallVector<std::unique_ptr<MemoryBuffer>, 4> Buffers;
> > +
> > + SmallVector<object::BuildIDRef> BinaryIDs;
> > + auto CoverageReadersOrErr = BinaryCoverageReader::create(
> > + CovMappingBufRef, Arch, Buffers, CompilationDir,
> > + FoundBinaryIDs ? &BinaryIDs : nullptr);
> > + if (Error E = CoverageReadersOrErr.takeError()) {
> > + E = handleMaybeNoDataFoundError(std::move(E));
> > + if (E)
> > + return createFileError(Filename, std::move(E));
> > + return E;
> > + }
> > +
> > + SmallVector<std::unique_ptr<CoverageMappingReader>, 4> Readers;
> > + for (auto &Reader : CoverageReadersOrErr.get())
> > + Readers.push_back(std::move(Reader));
> > + if (FoundBinaryIDs && !Readers.empty()) {
> > + llvm::append_range(*FoundBinaryIDs,
> > + llvm::map_range(BinaryIDs, [](object::BuildIDRef
> BID) {
> > + return object::BuildID(BID);
> > + }));
> > + }
> > + DataFound |= !Readers.empty();
> > + if (Error E = loadFromReaders(Readers, ProfileReader, Coverage))
> > + return createFileError(Filename, std::move(E));
> > + return Error::success();
> > +}
> > +
> > Expected<std::unique_ptr<CoverageMapping>>
> > CoverageMapping::load(ArrayRef<StringRef> ObjectFilenames,
> > StringRef ProfileFilename, ArrayRef<StringRef>
> Arches,
> > - StringRef CompilationDir) {
> > + StringRef CompilationDir,
> > + const object::BuildIDFetcher *BIDFetcher) {
> > auto ProfileReaderOrErr =
> IndexedInstrProfReader::create(ProfileFilename);
> > if (Error E = ProfileReaderOrErr.takeError())
> > return createFileError(ProfileFilename, std::move(E));
> > @@ -353,35 +393,56 @@ CoverageMapping::load(ArrayRef<StringRef>
> ObjectFilenames,
> > auto Coverage = std::unique_ptr<CoverageMapping>(new
> CoverageMapping());
> > bool DataFound = false;
> >
> > + auto GetArch = [&](size_t Idx) {
> > + if (Arches.empty())
> > + return StringRef();
> > + if (Arches.size() == 1)
> > + return Arches.front();
> > + return Arches[Idx];
> > + };
> > +
> > + SmallVector<object::BuildID> FoundBinaryIDs;
> > for (const auto &File : llvm::enumerate(ObjectFilenames)) {
> > - auto CovMappingBufOrErr = MemoryBuffer::getFileOrSTDIN(
> > - File.value(), /*IsText=*/false,
> /*RequiresNullTerminator=*/false);
> > - if (std::error_code EC = CovMappingBufOrErr.getError())
> > - return createFileError(File.value(), errorCodeToError(EC));
> > - StringRef Arch = Arches.empty() ? StringRef() :
> Arches[File.index()];
> > - MemoryBufferRef CovMappingBufRef =
> > - CovMappingBufOrErr.get()->getMemBufferRef();
> > - SmallVector<std::unique_ptr<MemoryBuffer>, 4> Buffers;
> > - auto CoverageReadersOrErr = BinaryCoverageReader::create(
> > - CovMappingBufRef, Arch, Buffers, CompilationDir);
> > - if (Error E = CoverageReadersOrErr.takeError()) {
> > - E = handleMaybeNoDataFoundError(std::move(E));
> > - if (E)
> > - return createFileError(File.value(), std::move(E));
> > - // E == success (originally a no_data_found error).
> > - continue;
> > + if (Error E =
> > + loadFromFile(File.value(), GetArch(File.index()),
> CompilationDir,
> > + *ProfileReader, *Coverage, DataFound,
> &FoundBinaryIDs))
> > + return std::move(E);
> > + }
> > +
> > + if (BIDFetcher) {
> > + const auto &Compare = [](object::BuildIDRef A, object::BuildIDRef
> B) {
> > + return StringRef(reinterpret_cast<const char *>(A.data()),
> A.size()) <
> > + StringRef(reinterpret_cast<const char *>(B.data()),
> B.size());
> > + };
> > + std::vector<object::BuildID> ProfileBinaryIDs;
> > + if (Error E = ProfileReader->readBinaryIds(ProfileBinaryIDs))
> > + return createFileError(ProfileFilename, std::move(E));
> > + llvm::sort(ProfileBinaryIDs, Compare);
> > + std::unique(ProfileBinaryIDs.begin(), ProfileBinaryIDs.end(),
> Compare);
> > +
> > + SmallVector<object::BuildIDRef> BinaryIDsToFetch;
> > + if (!ProfileBinaryIDs.empty()) {
> > + llvm::sort(FoundBinaryIDs, Compare);
> > + std::unique(FoundBinaryIDs.begin(), FoundBinaryIDs.end(),
> Compare);
> > + std::set_
> > diff erence(
> > + ProfileBinaryIDs.begin(), ProfileBinaryIDs.end(),
> > + FoundBinaryIDs.begin(), FoundBinaryIDs.end(),
> > + std::inserter(BinaryIDsToFetch, BinaryIDsToFetch.end()),
> Compare);
> > }
> >
> > - SmallVector<std::unique_ptr<CoverageMappingReader>, 4> Readers;
> > - for (auto &Reader : CoverageReadersOrErr.get())
> > - Readers.push_back(std::move(Reader));
> > - DataFound |= !Readers.empty();
> > - if (Error E = loadFromReaders(Readers, *ProfileReader, *Coverage))
> > - return createFileError(File.value(), std::move(E));
> > + for (object::BuildIDRef BinaryID : BinaryIDsToFetch) {
> > + std::optional<std::string> PathOpt = BIDFetcher->fetch(BinaryID);
> > + if (!PathOpt)
> > + continue;
> > + std::string Path = std::move(*PathOpt);
> > + StringRef Arch = Arches.size() == 1 ? Arches.front() :
> StringRef();
> > + if (Error E = loadFromFile(Path, Arch, CompilationDir,
> *ProfileReader,
> > + *Coverage, DataFound))
> > + return std::move(E);
> > + }
> > }
> > - // If no readers were created, either no objects were provided or
> none of them
> > - // had coverage data. Return an error in the latter case.
> > - if (!DataFound && !ObjectFilenames.empty())
> > +
> > + if (!DataFound)
> > return createFileError(
> > join(ObjectFilenames.begin(), ObjectFilenames.end(), ", "),
> > make_error<CoverageMapError>(coveragemap_error::no_data_found));
> >
> > diff --git a/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp
> b/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp
> > index 41962ab24ff9c..d313864e2ddb8 100644
> > --- a/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp
> > +++ b/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp
> > @@ -954,7 +954,8 @@ static Expected<std::vector<SectionRef>>
> lookupSections(ObjectFile &OF,
> >
> > static Expected<std::unique_ptr<BinaryCoverageReader>>
> > loadBinaryFormat(std::unique_ptr<Binary> Bin, StringRef Arch,
> > - StringRef CompilationDir = "") {
> > + StringRef CompilationDir = "",
> > + std::optional<object::BuildIDRef> *BinaryID = nullptr)
> {
> > std::unique_ptr<ObjectFile> OF;
> > if (auto *Universal = dyn_cast<MachOUniversalBinary>(Bin.get())) {
> > // If we have a universal binary, try to look up the object for the
> > @@ -1052,6 +1053,9 @@ loadBinaryFormat(std::unique_ptr<Binary> Bin,
> StringRef Arch,
> > FuncRecords = std::move(WritableBuffer);
> > }
> >
> > + if (BinaryID)
> > + *BinaryID = getBuildID(OF.get());
> > +
> > return BinaryCoverageReader::createCoverageReaderFromBuffer(
> > CoverageMapping, std::move(FuncRecords), std::move(ProfileNames),
> > BytesInAddress, Endian, CompilationDir);
> > @@ -1074,7 +1078,7 @@
> Expected<std::vector<std::unique_ptr<BinaryCoverageReader>>>
> > BinaryCoverageReader::create(
> > MemoryBufferRef ObjectBuffer, StringRef Arch,
> > SmallVectorImpl<std::unique_ptr<MemoryBuffer>> &ObjectFileBuffers,
> > - StringRef CompilationDir) {
> > + StringRef CompilationDir, SmallVectorImpl<object::BuildIDRef>
> *BinaryIDs) {
> > std::vector<std::unique_ptr<BinaryCoverageReader>> Readers;
> >
> > if (ObjectBuffer.getBuffer().startswith(TestingFormatMagic)) {
> > @@ -1114,7 +1118,7 @@ BinaryCoverageReader::create(
> >
> > return BinaryCoverageReader::create(
> > ArchiveOrErr.get()->getMemoryBufferRef(), Arch,
> ObjectFileBuffers,
> > - CompilationDir);
> > + CompilationDir, BinaryIDs);
> > }
> > }
> >
> > @@ -1127,7 +1131,8 @@ BinaryCoverageReader::create(
> > return ChildBufOrErr.takeError();
> >
> > auto ChildReadersOrErr = BinaryCoverageReader::create(
> > - ChildBufOrErr.get(), Arch, ObjectFileBuffers, CompilationDir);
> > + ChildBufOrErr.get(), Arch, ObjectFileBuffers, CompilationDir,
> > + BinaryIDs);
> > if (!ChildReadersOrErr)
> > return ChildReadersOrErr.takeError();
> > for (auto &Reader : ChildReadersOrErr.get())
> > @@ -1146,10 +1151,14 @@ BinaryCoverageReader::create(
> > return std::move(Readers);
> > }
> >
> > - auto ReaderOrErr = loadBinaryFormat(std::move(Bin), Arch,
> CompilationDir);
> > + std::optional<object::BuildIDRef> BinaryID;
> > + auto ReaderOrErr = loadBinaryFormat(std::move(Bin), Arch,
> CompilationDir,
> > + BinaryIDs ? &BinaryID : nullptr);
> > if (!ReaderOrErr)
> > return ReaderOrErr.takeError();
> > Readers.push_back(std::move(ReaderOrErr.get()));
> > + if (BinaryID)
> > + BinaryIDs->push_back(*BinaryID);
> > return std::move(Readers);
> > }
> >
> >
> > diff --git a/llvm/tools/llvm-cov/CMakeLists.txt
> b/llvm/tools/llvm-cov/CMakeLists.txt
> > index 300bbd0bc6dbd..7acc87e08a9ef 100644
> > --- a/llvm/tools/llvm-cov/CMakeLists.txt
> > +++ b/llvm/tools/llvm-cov/CMakeLists.txt
> > @@ -21,3 +21,5 @@ add_llvm_tool(llvm-cov
> > SourceCoverageViewText.cpp
> > TestingSupport.cpp
> > )
> > +
> > +target_link_libraries(llvm-cov PRIVATE LLVMDebuginfod)
> >
> > diff --git a/llvm/tools/llvm-cov/CodeCoverage.cpp
> b/llvm/tools/llvm-cov/CodeCoverage.cpp
> > index 2b2eda5d85873..7366059cd242f 100644
> > --- a/llvm/tools/llvm-cov/CodeCoverage.cpp
> > +++ b/llvm/tools/llvm-cov/CodeCoverage.cpp
> > @@ -23,6 +23,10 @@
> > #include "llvm/ADT/SmallString.h"
> > #include "llvm/ADT/StringRef.h"
> > #include "llvm/ADT/Triple.h"
> > +#include "llvm/Debuginfod/BuildIDFetcher.h"
> > +#include "llvm/Debuginfod/Debuginfod.h"
> > +#include "llvm/Debuginfod/HTTPClient.h"
> > +#include "llvm/Object/BuildID.h"
> > #include "llvm/ProfileData/Coverage/CoverageMapping.h"
> > #include "llvm/ProfileData/InstrProfReader.h"
> > #include "llvm/Support/CommandLine.h"
> > @@ -179,6 +183,8 @@ class CodeCoverageTool {
> >
> > /// Allowlist from -name-allowlist to be used for filtering.
> > std::unique_ptr<SpecialCaseList> NameAllowlist;
> > +
> > + std::unique_ptr<object::BuildIDFetcher> BIDFetcher;
> > };
> > }
> >
> > @@ -435,7 +441,7 @@ std::unique_ptr<CoverageMapping>
> CodeCoverageTool::load() {
> > ObjectFilename);
> > auto CoverageOrErr =
> > CoverageMapping::load(ObjectFilenames, PGOFilename,
> CoverageArches,
> > - ViewOpts.CompilationDirectory);
> > + ViewOpts.CompilationDirectory,
> BIDFetcher.get());
> > if (Error E = CoverageOrErr.takeError()) {
> > error("Failed to load coverage: " + toString(std::move(E)));
> > return nullptr;
> > @@ -647,6 +653,14 @@ int CodeCoverageTool::run(Command Cmd, int argc,
> const char **argv) {
> > cl::opt<bool> DebugDump("dump", cl::Optional,
> > cl::desc("Show internal debug dump"));
> >
> > + cl::list<std::string> DebugFileDirectory(
> > + "debug-file-directory",
> > + cl::desc("Directories to search for object files by build ID"));
> > + cl::opt<bool> Debuginfod(
> > + "debuginfod", cl::ZeroOrMore,
> > + cl::desc("Use debuginfod to look up object files from profile"),
> > + cl::init(canUseDebuginfod()));
> > +
> > cl::opt<CoverageViewOptions::OutputFormat> Format(
> > "format", cl::desc("Output format for line-based coverage
> reports"),
> > cl::values(clEnumValN(CoverageViewOptions::OutputFormat::Text,
> "text",
> > @@ -749,12 +763,18 @@ int CodeCoverageTool::run(Command Cmd, int argc,
> const char **argv) {
> > auto commandLineParser = [&, this](int argc, const char **argv) ->
> int {
> > cl::ParseCommandLineOptions(argc, argv, "LLVM code coverage
> tool\n");
> > ViewOpts.Debug = DebugDump;
> > + if (Debuginfod) {
> > + HTTPClient::initialize();
> > + BIDFetcher =
> std::make_unique<DebuginfodFetcher>(DebugFileDirectory);
> > + } else {
> > + BIDFetcher =
> std::make_unique<object::BuildIDFetcher>(DebugFileDirectory);
> > + }
> >
> > if (!CovFilename.empty())
> > ObjectFilenames.emplace_back(CovFilename);
> > for (const std::string &Filename : CovFilenames)
> > ObjectFilenames.emplace_back(Filename);
> > - if (ObjectFilenames.empty()) {
> > + if (ObjectFilenames.empty() && !Debuginfod &&
> DebugFileDirectory.empty()) {
> > errs() << "No filenames specified!\n";
> > ::exit(1);
> > }
> > @@ -867,10 +887,8 @@ int CodeCoverageTool::run(Command Cmd, int argc,
> const char **argv) {
> > }
> > CoverageArches.emplace_back(Arch);
> > }
> > - if (CoverageArches.size() == 1)
> > - CoverageArches.insert(CoverageArches.end(),
> ObjectFilenames.size() - 1,
> > - CoverageArches[0]);
> > - if (CoverageArches.size() != ObjectFilenames.size()) {
> > + if (CoverageArches.size() != 1 &&
> > + CoverageArches.size() != ObjectFilenames.size()) {
> > error("Number of architectures doesn't match the number of
> objects");
> > return 1;
> > }
> >
> > diff --git a/llvm/tools/llvm-objdump/llvm-objdump.cpp
> b/llvm/tools/llvm-objdump/llvm-objdump.cpp
> > index 930b132533cdd..9979a26cf1154 100644
> > --- a/llvm/tools/llvm-objdump/llvm-objdump.cpp
> > +++ b/llvm/tools/llvm-objdump/llvm-objdump.cpp
> > @@ -3198,9 +3198,7 @@ int main(int argc, char **argv) {
> >
> > // Initialize debuginfod.
> > const bool ShouldUseDebuginfodByDefault =
> > - InputArgs.hasArg(OBJDUMP_build_id) ||
> > - (HTTPClient::isAvailable() &&
> > - !ExitOnErr(getDefaultDebuginfodUrls()).empty());
> > + InputArgs.hasArg(OBJDUMP_build_id) || canUseDebuginfod();
> > std::vector<std::string> DebugFileDirectories =
> > InputArgs.getAllArgValues(OBJDUMP_debug_file_directory);
> > if (InputArgs.hasFlag(OBJDUMP_debuginfod, OBJDUMP_no_debuginfod,
> >
> > diff --git a/llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp
> b/llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp
> > index 1b86134dda516..ed24e85502911 100644
> > --- a/llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp
> > +++ b/llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp
> > @@ -443,13 +443,7 @@ int main(int argc, char **argv) {
> >
> > LLVMSymbolizer Symbolizer(Opts);
> >
> > - // A debuginfod lookup could succeed if a HTTP client is available
> and at
> > - // least one backing URL is configured.
> > - bool ShouldUseDebuginfodByDefault =
> > - HTTPClient::isAvailable() &&
> > - !ExitOnErr(getDefaultDebuginfodUrls()).empty();
> > - if (Args.hasFlag(OPT_debuginfod, OPT_no_debuginfod,
> > - ShouldUseDebuginfodByDefault))
> > + if (Args.hasFlag(OPT_debuginfod, OPT_no_debuginfod,
> canUseDebuginfod()))
> > enableDebuginfod(Symbolizer, Args);
> >
> > if (Args.hasArg(OPT_filter_markup)) {
> >
> >
> >
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
--
Daniel Thornburgh | dthorn at google.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230127/919f451f/attachment.html>
More information about the llvm-commits
mailing list