<div dir="ltr">Thanks, I sent a fix-forward for this. There was also a logic error with the comparator functions that this uncovered; unique takes ==, while sort and set_difference take <.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jan 27, 2023 at 6:37 AM Aaron Ballman <<a href="mailto:aaron@aaronballman.com">aaron@aaronballman.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">This appears to be causing new diagnostics in MSVC that should be addressed:<br>
<br>
Warning C4858 discarding return value: The 'unique' algorithm returns<br>
the iterator past the last element that should be kept. You need to<br>
call container.erase(result, container.end()) afterwards.<br>
F:\source\llvm-project\llvm\out\build\x64-Debug\llvm<br>
F:\source\llvm-project\llvm\lib\ProfileData\Coverage\CoverageMapping.cpp<br>
421<br>
Warning C4858 discarding return value: The 'unique' algorithm returns<br>
the iterator past the last element that should be kept. You need to<br>
call container.erase(result, container.end()) afterwards.<br>
F:\source\llvm-project\llvm\out\build\x64-Debug\llvm<br>
F:\source\llvm-project\llvm\lib\ProfileData\Coverage\CoverageMapping.cpp<br>
426<br>
<br>
~Aaron<br>
<br>
On Thu, Jan 26, 2023 at 3:59 PM Daniel Thornburgh via llvm-commits<br>
<<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>> wrote:<br>
><br>
><br>
> Author: Daniel Thornburgh<br>
> Date: 2023-01-26T12:59:52-08:00<br>
> New Revision: a3b0dde4edb93248f74d86c63b0ae108854004e6<br>
><br>
> URL: <a href="https://github.com/llvm/llvm-project/commit/a3b0dde4edb93248f74d86c63b0ae108854004e6" rel="noreferrer" target="_blank">https://github.com/llvm/llvm-project/commit/a3b0dde4edb93248f74d86c63b0ae108854004e6</a><br>
> DIFF: <a href="https://github.com/llvm/llvm-project/commit/a3b0dde4edb93248f74d86c63b0ae108854004e6.diff" rel="noreferrer" target="_blank">https://github.com/llvm/llvm-project/commit/a3b0dde4edb93248f74d86c63b0ae108854004e6.diff</a><br>
><br>
> LOG: Reland: [llvm-cov] Look up object files using debuginfod<br>
><br>
> Reviewed By: gulfem<br>
><br>
> Differential Revision: <a href="https://reviews.llvm.org/D136702" rel="noreferrer" target="_blank">https://reviews.llvm.org/D136702</a><br>
><br>
> Added:<br>
> compiler-rt/test/profile/Linux/binary-id-debuginfod.c<br>
> compiler-rt/test/profile/Linux/binary-id-lookup.c<br>
><br>
> Modified:<br>
> compiler-rt/test/profile/CMakeLists.txt<br>
> compiler-rt/test/profile/Linux/<a href="http://lit.local.cfg.py" rel="noreferrer" target="_blank">lit.local.cfg.py</a><br>
> compiler-rt/test/profile/<a href="http://lit.site.cfg.py.in" rel="noreferrer" target="_blank">lit.site.cfg.py.in</a><br>
> llvm/cmake/modules/<a href="http://LLVMConfig.cmake.in" rel="noreferrer" target="_blank">LLVMConfig.cmake.in</a><br>
> llvm/docs/CommandGuide/llvm-cov.rst<br>
> llvm/include/llvm/Debuginfod/Debuginfod.h<br>
> llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h<br>
> llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h<br>
> llvm/lib/Debuginfod/Debuginfod.cpp<br>
> llvm/lib/ProfileData/Coverage/CoverageMapping.cpp<br>
> llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp<br>
> llvm/tools/llvm-cov/CMakeLists.txt<br>
> llvm/tools/llvm-cov/CodeCoverage.cpp<br>
> llvm/tools/llvm-objdump/llvm-objdump.cpp<br>
> llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp<br>
><br>
> Removed:<br>
><br>
><br>
><br>
> ################################################################################<br>
> diff --git a/compiler-rt/test/profile/CMakeLists.txt b/compiler-rt/test/profile/CMakeLists.txt<br>
> index 782be305a13ca..1b61d1ea0728e 100644<br>
> --- a/compiler-rt/test/profile/CMakeLists.txt<br>
> +++ b/compiler-rt/test/profile/CMakeLists.txt<br>
> @@ -16,6 +16,8 @@ if(APPLE)<br>
> darwin_filter_host_archs(PROFILE_SUPPORTED_ARCH PROFILE_TEST_ARCH)<br>
> endif()<br>
><br>
> +pythonize_bool(LLVM_ENABLE_CURL)<br>
> +<br>
> foreach(arch ${PROFILE_TEST_ARCH})<br>
> set(PROFILE_TEST_TARGET_ARCH ${arch})<br>
> get_test_cc_for_arch(${arch} PROFILE_TEST_TARGET_CC PROFILE_TEST_TARGET_CFLAGS)<br>
><br>
> diff --git a/compiler-rt/test/profile/Linux/binary-id-debuginfod.c b/compiler-rt/test/profile/Linux/binary-id-debuginfod.c<br>
> new file mode 100644<br>
> index 0000000000000..39a125b8c09f7<br>
> --- /dev/null<br>
> +++ b/compiler-rt/test/profile/Linux/binary-id-debuginfod.c<br>
> @@ -0,0 +1,33 @@<br>
> +// REQUIRES: linux, curl<br>
> +// RUN: split-file %s %t<br>
> +// RUN: %clang_profgen -Wl,--build-id=0x12345678 -fcoverage-mapping -O2 -shared %t/foo.c -o %t/libfoo.so<br>
> +// RUN: %clang_profgen -Wl,--build-id=0xabcd1234 -fcoverage-mapping -O2 %t/main.c -L%t -lfoo -o %t.main<br>
> +// RUN: rm -rf %t.profdir<br>
> +// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw LD_LIBRARY_PATH=%t %run %t.main<br>
> +// RUN: mkdir -p %t/buildid/12345678 %t/buildid/abcd1234<br>
> +// RUN: cp %t/libfoo.so %t/buildid/12345678/debuginfo<br>
> +// RUN: cp %t.main %t/buildid/abcd1234/debuginfo<br>
> +// RUN: llvm-profdata merge -o %t.profdata %t.profdir/default_*.profraw<br>
> +// RUN: env DEBUGINFOD_URLS=file://%t llvm-cov show -instr-profile %t.profdata | FileCheck %s<br>
> +// RUN: echo "bad" > %t/libfoo.so %t/buildid/12345678/debuginfo<br>
> +// RUN: echo "bad" > %t/buildid/abcd1234/debuginfo<br>
> +// RUN: env DEBUGINFOD_URLS=file://%t llvm-cov show -instr-profile %t.profdata -debuginfod=false %t.main | FileCheck %s --check-prefix=NODEBUGINFOD<br>
> +<br>
> +// CHECK: 1| 1|void foo(void) {}<br>
> +// CHECK: 2| 1|void bar(void) {}<br>
> +// CHECK: 3| 1|int main() {<br>
> +<br>
> +// NODEBUGINFOD-NOT: foo(void) {}<br>
> +// NODEBUGINFOD: main<br>
> +<br>
> +//--- foo.c<br>
> +void foo(void) {}<br>
> +<br>
> +//--- main.c<br>
> +void foo(void);<br>
> +void bar(void) {}<br>
> +int main() {<br>
> + foo();<br>
> + bar();<br>
> + return 0;<br>
> +}<br>
><br>
> diff --git a/compiler-rt/test/profile/Linux/binary-id-lookup.c b/compiler-rt/test/profile/Linux/binary-id-lookup.c<br>
> new file mode 100644<br>
> index 0000000000000..9cc6648903e1d<br>
> --- /dev/null<br>
> +++ b/compiler-rt/test/profile/Linux/binary-id-lookup.c<br>
> @@ -0,0 +1,32 @@<br>
> +// REQUIRES: linux<br>
> +// RUN: split-file %s %t<br>
> +// RUN: %clang_profgen -Wl,--build-id=0x12345678 -fcoverage-mapping -O2 -shared %t/foo.c -o %t/libfoo.so<br>
> +// RUN: %clang_profgen -Wl,--build-id=0xabcd1234 -fcoverage-mapping -O2 %t/main.c -L%t -lfoo -o %t.main<br>
> +// RUN: rm -rf %t.profdir<br>
> +// RUN: env LLVM_PROFILE_FILE=%t.profdir/default_%m.profraw LD_LIBRARY_PATH=%t %run %t.main<br>
> +// RUN: mkdir -p %t/.build-id/12 %t/.build-id/ab<br>
> +// RUN: cp %t/libfoo.so %t/.build-id/12/345678.debug<br>
> +// RUN: cp %t.main %t/.build-id/ab/cd1234.debug<br>
> +// RUN: llvm-profdata merge -o %t.profdata %t.profdir/default_*.profraw<br>
> +// RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory %t | FileCheck %s<br>
> +// RUN: echo "bad" > %t/.build-id/ab/cd1234.debug<br>
> +// RUN: llvm-cov show -instr-profile %t.profdata -debug-file-directory %t %t.main | FileCheck %s<br>
> +// RUN: not llvm-cov show -instr-profile %t.profdata -debug-file-directory %t/empty 2>&1 | FileCheck %s --check-prefix=NODATA<br>
> +<br>
> +// CHECK: 1| 1|void foo(void) {}<br>
> +// CHECK: 2| 1|void bar(void) {}<br>
> +// CHECK: 3| 1|int main() {<br>
> +<br>
> +// NODATA: error: Failed to load coverage: '': No coverage data found<br>
> +<br>
> +//--- foo.c<br>
> +void foo(void) {}<br>
> +<br>
> +//--- main.c<br>
> +void foo(void);<br>
> +void bar(void) {}<br>
> +int main() {<br>
> + foo();<br>
> + bar();<br>
> + return 0;<br>
> +}<br>
><br>
> diff --git a/compiler-rt/test/profile/Linux/<a href="http://lit.local.cfg.py" rel="noreferrer" target="_blank">lit.local.cfg.py</a> b/compiler-rt/test/profile/Linux/<a href="http://lit.local.cfg.py" rel="noreferrer" target="_blank">lit.local.cfg.py</a><br>
> index 98d79518b92aa..9bb92b7f14fe6 100644<br>
> --- a/compiler-rt/test/profile/Linux/<a href="http://lit.local.cfg.py" rel="noreferrer" target="_blank">lit.local.cfg.py</a><br>
> +++ b/compiler-rt/test/profile/Linux/<a href="http://lit.local.cfg.py" rel="noreferrer" target="_blank">lit.local.cfg.py</a><br>
> @@ -38,3 +38,6 @@ def is_gold_linker_available():<br>
><br>
> if root.host_os not in ['Linux'] or not is_gold_linker_available():<br>
> config.unsupported = True<br>
> +<br>
> +if config.have_curl:<br>
> + config.available_features.add('curl')<br>
><br>
> diff --git a/compiler-rt/test/profile/<a href="http://lit.site.cfg.py.in" rel="noreferrer" target="_blank">lit.site.cfg.py.in</a> b/compiler-rt/test/profile/<a href="http://lit.site.cfg.py.in" rel="noreferrer" target="_blank">lit.site.cfg.py.in</a><br>
> index bb81fb0c93f75..3466eabf7248d 100644<br>
> --- a/compiler-rt/test/profile/<a href="http://lit.site.cfg.py.in" rel="noreferrer" target="_blank">lit.site.cfg.py.in</a><br>
> +++ b/compiler-rt/test/profile/<a href="http://lit.site.cfg.py.in" rel="noreferrer" target="_blank">lit.site.cfg.py.in</a><br>
> @@ -4,6 +4,7 @@<br>
> config.profile_lit_binary_dir = "@PROFILE_LIT_BINARY_DIR@"<br>
> config.target_cflags = "@PROFILE_TEST_TARGET_CFLAGS@"<br>
> config.target_arch = "@PROFILE_TEST_TARGET_ARCH@"<br>
> +config.have_curl = @LLVM_ENABLE_CURL_PYBOOL@<br>
><br>
> # Load common config for all compiler-rt lit tests.<br>
> lit_config.load_config(config, "@COMPILER_RT_BINARY_DIR@/test/lit.common.configured")<br>
><br>
> diff --git a/llvm/cmake/modules/<a href="http://LLVMConfig.cmake.in" rel="noreferrer" target="_blank">LLVMConfig.cmake.in</a> b/llvm/cmake/modules/<a href="http://LLVMConfig.cmake.in" rel="noreferrer" target="_blank">LLVMConfig.cmake.in</a><br>
> index bae2a07c0fb16..2d90512bfb06e 100644<br>
> --- a/llvm/cmake/modules/<a href="http://LLVMConfig.cmake.in" rel="noreferrer" target="_blank">LLVMConfig.cmake.in</a><br>
> +++ b/llvm/cmake/modules/<a href="http://LLVMConfig.cmake.in" rel="noreferrer" target="_blank">LLVMConfig.cmake.in</a><br>
> @@ -83,6 +83,11 @@ if(LLVM_ENABLE_LIBXML2)<br>
> find_package(LibXml2)<br>
> endif()<br>
><br>
> +set(LLVM_ENABLE_CURL @LLVM_ENABLE_CURL@)<br>
> +if(LLVM_ENABLE_CURL)<br>
> + find_package(CURL)<br>
> +endif()<br>
> +<br>
> set(LLVM_WITH_Z3 @LLVM_WITH_Z3@)<br>
><br>
> set(LLVM_ENABLE_DIA_SDK @LLVM_ENABLE_DIA_SDK@)<br>
><br>
> diff --git a/llvm/docs/CommandGuide/llvm-cov.rst b/llvm/docs/CommandGuide/llvm-cov.rst<br>
> index 319835de168a9..495faa7201e18 100644<br>
> --- a/llvm/docs/CommandGuide/llvm-cov.rst<br>
> +++ b/llvm/docs/CommandGuide/llvm-cov.rst<br>
> @@ -349,6 +349,17 @@ OPTIONS<br>
> coverage >= high, red when coverage < low, and yellow otherwise. Both high and<br>
> low should be between 0-100 and high > low.<br>
><br>
> +.. option:: -debuginfod<br>
> +<br>
> +Use debuginfod to look up coverage mapping for binary IDs present in the profile<br>
> +but not in any object given on the command line. Defaults to true if debuginfod<br>
> +is compiled in and configured via the DEBUGINFOD_URLS environment variable.<br>
> +<br>
> +.. option:: -debug-file-directory=<dir><br>
> +<br>
> +Provides local directories to search for objects corresponding to binary IDs in<br>
> +the profile (as with debuginfod). Defaults to system build ID directories.<br>
> +<br>
> .. program:: llvm-cov report<br>
><br>
> .. _llvm-cov-report:<br>
> @@ -418,6 +429,18 @@ OPTIONS<br>
> when binaries have been compiled with one of `-fcoverage-prefix-map`<br>
> `-fcoverage-compilation-dir`, or `-ffile-compilation-dir`.<br>
><br>
> +.. option:: -debuginfod<br>
> +<br>
> +Attempt to look up coverage mapping from objects using debuginfod. This is<br>
> +attempted by default for binary IDs present in the profile but not provided on<br>
> +the command line, so long as debuginfod is compiled in and configured via<br>
> +DEBUGINFOD_URLS.<br>
> +<br>
> +.. option:: -debug-file-directory=<dir><br>
> +<br>
> +Provides a directory to search for objects corresponding to binary IDs in the<br>
> +profile.<br>
> +<br>
> .. program:: llvm-cov export<br>
><br>
> .. _llvm-cov-export:<br>
> @@ -492,3 +515,15 @@ OPTIONS<br>
> Directory used as a base for relative coverage mapping paths. Only applicable<br>
> when binaries have been compiled with one of `-fcoverage-prefix-map`<br>
> `-fcoverage-compilation-dir`, or `-ffile-compilation-dir`.<br>
> +<br>
> +.. option:: -debuginfod<br>
> +<br>
> +Attempt to look up coverage mapping from objects using debuginfod. This is<br>
> +attempted by default for binary IDs present in the profile but not provided on<br>
> +the command line, so long as debuginfod is compiled in and configured via<br>
> +DEBUGINFOD_URLS.<br>
> +<br>
> +.. option:: -debug-file-directory=<dir><br>
> +<br>
> +Provides a directory to search for objects corresponding to binary IDs in the<br>
> +profile.<br>
><br>
> diff --git a/llvm/include/llvm/Debuginfod/Debuginfod.h b/llvm/include/llvm/Debuginfod/Debuginfod.h<br>
> index caece0e6fc194..ec7f5691dda4f 100644<br>
> --- a/llvm/include/llvm/Debuginfod/Debuginfod.h<br>
> +++ b/llvm/include/llvm/Debuginfod/Debuginfod.h<br>
> @@ -38,9 +38,13 @@<br>
><br>
> namespace llvm {<br>
><br>
> +/// Returns false if a debuginfod lookup can be determined to have no chance of<br>
> +/// succeeding.<br>
> +bool canUseDebuginfod();<br>
> +<br>
> /// Finds default array of Debuginfod server URLs by checking DEBUGINFOD_URLS<br>
> /// environment variable.<br>
> -Expected<SmallVector<StringRef>> getDefaultDebuginfodUrls();<br>
> +SmallVector<StringRef> getDefaultDebuginfodUrls();<br>
><br>
> /// Finds a default local file caching directory for the debuginfod client,<br>
> /// first checking DEBUGINFOD_CACHE_PATH.<br>
><br>
> diff --git a/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h b/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h<br>
> index 4d48308d5509e..bdb7728624686 100644<br>
> --- a/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h<br>
> +++ b/llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h<br>
> @@ -21,6 +21,7 @@<br>
> #include "llvm/ADT/StringRef.h"<br>
> #include "llvm/ADT/iterator.h"<br>
> #include "llvm/ADT/iterator_range.h"<br>
> +#include "llvm/Object/BuildID.h"<br>
> #include "llvm/ProfileData/InstrProf.h"<br>
> #include "llvm/Support/Alignment.h"<br>
> #include "llvm/Support/Compiler.h"<br>
> @@ -42,6 +43,10 @@ namespace llvm {<br>
><br>
> class IndexedInstrProfReader;<br>
><br>
> +namespace object {<br>
> +class BuildIDFetcher;<br>
> +} // namespace object<br>
> +<br>
> namespace coverage {<br>
><br>
> class CoverageMappingReader;<br>
> @@ -579,6 +584,13 @@ class CoverageMapping {<br>
> ArrayRef<std::unique_ptr<CoverageMappingReader>> CoverageReaders,<br>
> IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage);<br>
><br>
> + // Load coverage records from file.<br>
> + static Error<br>
> + loadFromFile(StringRef Filename, StringRef Arch, StringRef CompilationDir,<br>
> + IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage,<br>
> + bool &DataFound,<br>
> + SmallVectorImpl<object::BuildID> *FoundBinaryIDs = nullptr);<br>
> +<br>
> /// Add a function record corresponding to \p Record.<br>
> Error loadFunctionRecord(const CoverageMappingRecord &Record,<br>
> IndexedInstrProfReader &ProfileReader);<br>
> @@ -604,8 +616,8 @@ class CoverageMapping {<br>
> /// Ignores non-instrumented object files unless all are not instrumented.<br>
> static Expected<std::unique_ptr<CoverageMapping>><br>
> load(ArrayRef<StringRef> ObjectFilenames, StringRef ProfileFilename,<br>
> - ArrayRef<StringRef> Arches = std::nullopt,<br>
> - StringRef CompilationDir = "");<br>
> + ArrayRef<StringRef> Arches = std::nullopt, StringRef CompilationDir = "",<br>
> + const object::BuildIDFetcher *BIDFetcher = nullptr);<br>
><br>
> /// The number of functions that couldn't have their profiles mapped.<br>
> ///<br>
><br>
> diff --git a/llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h b/llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h<br>
> index 39c0045369be7..326c1b0d33384 100644<br>
> --- a/llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h<br>
> +++ b/llvm/include/llvm/ProfileData/Coverage/CoverageMappingReader.h<br>
> @@ -205,7 +205,8 @@ class BinaryCoverageReader : public CoverageMappingReader {<br>
> static Expected<std::vector<std::unique_ptr<BinaryCoverageReader>>><br>
> create(MemoryBufferRef ObjectBuffer, StringRef Arch,<br>
> SmallVectorImpl<std::unique_ptr<MemoryBuffer>> &ObjectFileBuffers,<br>
> - StringRef CompilationDir = "");<br>
> + StringRef CompilationDir = "",<br>
> + SmallVectorImpl<object::BuildIDRef> *BinaryIDs = nullptr);<br>
><br>
> static Expected<std::unique_ptr<BinaryCoverageReader>><br>
> createCoverageReaderFromBuffer(StringRef Coverage,<br>
><br>
> diff --git a/llvm/lib/Debuginfod/Debuginfod.cpp b/llvm/lib/Debuginfod/Debuginfod.cpp<br>
> index 026f118bbf5b5..2b0710b536bae 100644<br>
> --- a/llvm/lib/Debuginfod/Debuginfod.cpp<br>
> +++ b/llvm/lib/Debuginfod/Debuginfod.cpp<br>
> @@ -55,7 +55,11 @@ static std::string buildIDToString(BuildIDRef ID) {<br>
> return llvm::toHex(ID, /*LowerCase=*/true);<br>
> }<br>
><br>
> -Expected<SmallVector<StringRef>> getDefaultDebuginfodUrls() {<br>
> +bool canUseDebuginfod() {<br>
> + return HTTPClient::isAvailable() && !getDefaultDebuginfodUrls().empty();<br>
> +}<br>
> +<br>
> +SmallVector<StringRef> getDefaultDebuginfodUrls() {<br>
> const char *DebuginfodUrlsEnv = std::getenv("DEBUGINFOD_URLS");<br>
> if (DebuginfodUrlsEnv == nullptr)<br>
> return SmallVector<StringRef>();<br>
> @@ -126,13 +130,8 @@ Expected<std::string> getCachedOrDownloadArtifact(StringRef UniqueKey,<br>
> return CacheDirOrErr.takeError();<br>
> CacheDir = *CacheDirOrErr;<br>
><br>
> - Expected<SmallVector<StringRef>> DebuginfodUrlsOrErr =<br>
> - getDefaultDebuginfodUrls();<br>
> - if (!DebuginfodUrlsOrErr)<br>
> - return DebuginfodUrlsOrErr.takeError();<br>
> - SmallVector<StringRef> &DebuginfodUrls = *DebuginfodUrlsOrErr;<br>
> return getCachedOrDownloadArtifact(UniqueKey, UrlPath, CacheDir,<br>
> - DebuginfodUrls,<br>
> + getDefaultDebuginfodUrls(),<br>
> getDefaultDebuginfodTimeout());<br>
> }<br>
><br>
> @@ -159,7 +158,8 @@ class StreamedHTTPResponseHandler : public HTTPResponseHandler {<br>
><br>
> Error StreamedHTTPResponseHandler::handleBodyChunk(StringRef BodyChunk) {<br>
> if (!FileStream) {<br>
> - if (Client.responseCode() != 200)<br>
> + unsigned Code = Client.responseCode();<br>
> + if (Code && Code != 200)<br>
> return Error::success();<br>
> Expected<std::unique_ptr<CachedFileStream>> FileStreamOrError =<br>
> CreateStream();<br>
> @@ -259,7 +259,8 @@ Expected<std::string> getCachedOrDownloadArtifact(<br>
> if (Err)<br>
> return std::move(Err);<br>
><br>
> - if (Client.responseCode() != 200)<br>
> + unsigned Code = Client.responseCode();<br>
> + if (Code && Code != 200)<br>
> continue;<br>
><br>
> // Return the path to the artifact on disk.<br>
><br>
> diff --git a/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp b/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp<br>
> index 6113f78aeb4ee..9e3a3d7ba40fc 100644<br>
> --- a/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp<br>
> +++ b/llvm/lib/ProfileData/Coverage/CoverageMapping.cpp<br>
> @@ -17,6 +17,7 @@<br>
> #include "llvm/ADT/SmallBitVector.h"<br>
> #include "llvm/ADT/SmallVector.h"<br>
> #include "llvm/ADT/StringRef.h"<br>
> +#include "llvm/Object/BuildID.h"<br>
> #include "llvm/ProfileData/Coverage/CoverageMappingReader.h"<br>
> #include "llvm/ProfileData/InstrProfReader.h"<br>
> #include "llvm/Support/Debug.h"<br>
> @@ -342,10 +343,49 @@ static Error handleMaybeNoDataFoundError(Error E) {<br>
> });<br>
> }<br>
><br>
> +Error CoverageMapping::loadFromFile(<br>
> + StringRef Filename, StringRef Arch, StringRef CompilationDir,<br>
> + IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage,<br>
> + bool &DataFound, SmallVectorImpl<object::BuildID> *FoundBinaryIDs) {<br>
> + auto CovMappingBufOrErr = MemoryBuffer::getFileOrSTDIN(<br>
> + Filename, /*IsText=*/false, /*RequiresNullTerminator=*/false);<br>
> + if (std::error_code EC = CovMappingBufOrErr.getError())<br>
> + return createFileError(Filename, errorCodeToError(EC));<br>
> + MemoryBufferRef CovMappingBufRef =<br>
> + CovMappingBufOrErr.get()->getMemBufferRef();<br>
> + SmallVector<std::unique_ptr<MemoryBuffer>, 4> Buffers;<br>
> +<br>
> + SmallVector<object::BuildIDRef> BinaryIDs;<br>
> + auto CoverageReadersOrErr = BinaryCoverageReader::create(<br>
> + CovMappingBufRef, Arch, Buffers, CompilationDir,<br>
> + FoundBinaryIDs ? &BinaryIDs : nullptr);<br>
> + if (Error E = CoverageReadersOrErr.takeError()) {<br>
> + E = handleMaybeNoDataFoundError(std::move(E));<br>
> + if (E)<br>
> + return createFileError(Filename, std::move(E));<br>
> + return E;<br>
> + }<br>
> +<br>
> + SmallVector<std::unique_ptr<CoverageMappingReader>, 4> Readers;<br>
> + for (auto &Reader : CoverageReadersOrErr.get())<br>
> + Readers.push_back(std::move(Reader));<br>
> + if (FoundBinaryIDs && !Readers.empty()) {<br>
> + llvm::append_range(*FoundBinaryIDs,<br>
> + llvm::map_range(BinaryIDs, [](object::BuildIDRef BID) {<br>
> + return object::BuildID(BID);<br>
> + }));<br>
> + }<br>
> + DataFound |= !Readers.empty();<br>
> + if (Error E = loadFromReaders(Readers, ProfileReader, Coverage))<br>
> + return createFileError(Filename, std::move(E));<br>
> + return Error::success();<br>
> +}<br>
> +<br>
> Expected<std::unique_ptr<CoverageMapping>><br>
> CoverageMapping::load(ArrayRef<StringRef> ObjectFilenames,<br>
> StringRef ProfileFilename, ArrayRef<StringRef> Arches,<br>
> - StringRef CompilationDir) {<br>
> + StringRef CompilationDir,<br>
> + const object::BuildIDFetcher *BIDFetcher) {<br>
> auto ProfileReaderOrErr = IndexedInstrProfReader::create(ProfileFilename);<br>
> if (Error E = ProfileReaderOrErr.takeError())<br>
> return createFileError(ProfileFilename, std::move(E));<br>
> @@ -353,35 +393,56 @@ CoverageMapping::load(ArrayRef<StringRef> ObjectFilenames,<br>
> auto Coverage = std::unique_ptr<CoverageMapping>(new CoverageMapping());<br>
> bool DataFound = false;<br>
><br>
> + auto GetArch = [&](size_t Idx) {<br>
> + if (Arches.empty())<br>
> + return StringRef();<br>
> + if (Arches.size() == 1)<br>
> + return Arches.front();<br>
> + return Arches[Idx];<br>
> + };<br>
> +<br>
> + SmallVector<object::BuildID> FoundBinaryIDs;<br>
> for (const auto &File : llvm::enumerate(ObjectFilenames)) {<br>
> - auto CovMappingBufOrErr = MemoryBuffer::getFileOrSTDIN(<br>
> - File.value(), /*IsText=*/false, /*RequiresNullTerminator=*/false);<br>
> - if (std::error_code EC = CovMappingBufOrErr.getError())<br>
> - return createFileError(File.value(), errorCodeToError(EC));<br>
> - StringRef Arch = Arches.empty() ? StringRef() : Arches[File.index()];<br>
> - MemoryBufferRef CovMappingBufRef =<br>
> - CovMappingBufOrErr.get()->getMemBufferRef();<br>
> - SmallVector<std::unique_ptr<MemoryBuffer>, 4> Buffers;<br>
> - auto CoverageReadersOrErr = BinaryCoverageReader::create(<br>
> - CovMappingBufRef, Arch, Buffers, CompilationDir);<br>
> - if (Error E = CoverageReadersOrErr.takeError()) {<br>
> - E = handleMaybeNoDataFoundError(std::move(E));<br>
> - if (E)<br>
> - return createFileError(File.value(), std::move(E));<br>
> - // E == success (originally a no_data_found error).<br>
> - continue;<br>
> + if (Error E =<br>
> + loadFromFile(File.value(), GetArch(File.index()), CompilationDir,<br>
> + *ProfileReader, *Coverage, DataFound, &FoundBinaryIDs))<br>
> + return std::move(E);<br>
> + }<br>
> +<br>
> + if (BIDFetcher) {<br>
> + const auto &Compare = [](object::BuildIDRef A, object::BuildIDRef B) {<br>
> + return StringRef(reinterpret_cast<const char *>(A.data()), A.size()) <<br>
> + StringRef(reinterpret_cast<const char *>(B.data()), B.size());<br>
> + };<br>
> + std::vector<object::BuildID> ProfileBinaryIDs;<br>
> + if (Error E = ProfileReader->readBinaryIds(ProfileBinaryIDs))<br>
> + return createFileError(ProfileFilename, std::move(E));<br>
> + llvm::sort(ProfileBinaryIDs, Compare);<br>
> + std::unique(ProfileBinaryIDs.begin(), ProfileBinaryIDs.end(), Compare);<br>
> +<br>
> + SmallVector<object::BuildIDRef> BinaryIDsToFetch;<br>
> + if (!ProfileBinaryIDs.empty()) {<br>
> + llvm::sort(FoundBinaryIDs, Compare);<br>
> + std::unique(FoundBinaryIDs.begin(), FoundBinaryIDs.end(), Compare);<br>
> + std::set_<br>
> diff erence(<br>
> + ProfileBinaryIDs.begin(), ProfileBinaryIDs.end(),<br>
> + FoundBinaryIDs.begin(), FoundBinaryIDs.end(),<br>
> + std::inserter(BinaryIDsToFetch, BinaryIDsToFetch.end()), Compare);<br>
> }<br>
><br>
> - SmallVector<std::unique_ptr<CoverageMappingReader>, 4> Readers;<br>
> - for (auto &Reader : CoverageReadersOrErr.get())<br>
> - Readers.push_back(std::move(Reader));<br>
> - DataFound |= !Readers.empty();<br>
> - if (Error E = loadFromReaders(Readers, *ProfileReader, *Coverage))<br>
> - return createFileError(File.value(), std::move(E));<br>
> + for (object::BuildIDRef BinaryID : BinaryIDsToFetch) {<br>
> + std::optional<std::string> PathOpt = BIDFetcher->fetch(BinaryID);<br>
> + if (!PathOpt)<br>
> + continue;<br>
> + std::string Path = std::move(*PathOpt);<br>
> + StringRef Arch = Arches.size() == 1 ? Arches.front() : StringRef();<br>
> + if (Error E = loadFromFile(Path, Arch, CompilationDir, *ProfileReader,<br>
> + *Coverage, DataFound))<br>
> + return std::move(E);<br>
> + }<br>
> }<br>
> - // If no readers were created, either no objects were provided or none of them<br>
> - // had coverage data. Return an error in the latter case.<br>
> - if (!DataFound && !ObjectFilenames.empty())<br>
> +<br>
> + if (!DataFound)<br>
> return createFileError(<br>
> join(ObjectFilenames.begin(), ObjectFilenames.end(), ", "),<br>
> make_error<CoverageMapError>(coveragemap_error::no_data_found));<br>
><br>
> diff --git a/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp b/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp<br>
> index 41962ab24ff9c..d313864e2ddb8 100644<br>
> --- a/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp<br>
> +++ b/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp<br>
> @@ -954,7 +954,8 @@ static Expected<std::vector<SectionRef>> lookupSections(ObjectFile &OF,<br>
><br>
> static Expected<std::unique_ptr<BinaryCoverageReader>><br>
> loadBinaryFormat(std::unique_ptr<Binary> Bin, StringRef Arch,<br>
> - StringRef CompilationDir = "") {<br>
> + StringRef CompilationDir = "",<br>
> + std::optional<object::BuildIDRef> *BinaryID = nullptr) {<br>
> std::unique_ptr<ObjectFile> OF;<br>
> if (auto *Universal = dyn_cast<MachOUniversalBinary>(Bin.get())) {<br>
> // If we have a universal binary, try to look up the object for the<br>
> @@ -1052,6 +1053,9 @@ loadBinaryFormat(std::unique_ptr<Binary> Bin, StringRef Arch,<br>
> FuncRecords = std::move(WritableBuffer);<br>
> }<br>
><br>
> + if (BinaryID)<br>
> + *BinaryID = getBuildID(OF.get());<br>
> +<br>
> return BinaryCoverageReader::createCoverageReaderFromBuffer(<br>
> CoverageMapping, std::move(FuncRecords), std::move(ProfileNames),<br>
> BytesInAddress, Endian, CompilationDir);<br>
> @@ -1074,7 +1078,7 @@ Expected<std::vector<std::unique_ptr<BinaryCoverageReader>>><br>
> BinaryCoverageReader::create(<br>
> MemoryBufferRef ObjectBuffer, StringRef Arch,<br>
> SmallVectorImpl<std::unique_ptr<MemoryBuffer>> &ObjectFileBuffers,<br>
> - StringRef CompilationDir) {<br>
> + StringRef CompilationDir, SmallVectorImpl<object::BuildIDRef> *BinaryIDs) {<br>
> std::vector<std::unique_ptr<BinaryCoverageReader>> Readers;<br>
><br>
> if (ObjectBuffer.getBuffer().startswith(TestingFormatMagic)) {<br>
> @@ -1114,7 +1118,7 @@ BinaryCoverageReader::create(<br>
><br>
> return BinaryCoverageReader::create(<br>
> ArchiveOrErr.get()->getMemoryBufferRef(), Arch, ObjectFileBuffers,<br>
> - CompilationDir);<br>
> + CompilationDir, BinaryIDs);<br>
> }<br>
> }<br>
><br>
> @@ -1127,7 +1131,8 @@ BinaryCoverageReader::create(<br>
> return ChildBufOrErr.takeError();<br>
><br>
> auto ChildReadersOrErr = BinaryCoverageReader::create(<br>
> - ChildBufOrErr.get(), Arch, ObjectFileBuffers, CompilationDir);<br>
> + ChildBufOrErr.get(), Arch, ObjectFileBuffers, CompilationDir,<br>
> + BinaryIDs);<br>
> if (!ChildReadersOrErr)<br>
> return ChildReadersOrErr.takeError();<br>
> for (auto &Reader : ChildReadersOrErr.get())<br>
> @@ -1146,10 +1151,14 @@ BinaryCoverageReader::create(<br>
> return std::move(Readers);<br>
> }<br>
><br>
> - auto ReaderOrErr = loadBinaryFormat(std::move(Bin), Arch, CompilationDir);<br>
> + std::optional<object::BuildIDRef> BinaryID;<br>
> + auto ReaderOrErr = loadBinaryFormat(std::move(Bin), Arch, CompilationDir,<br>
> + BinaryIDs ? &BinaryID : nullptr);<br>
> if (!ReaderOrErr)<br>
> return ReaderOrErr.takeError();<br>
> Readers.push_back(std::move(ReaderOrErr.get()));<br>
> + if (BinaryID)<br>
> + BinaryIDs->push_back(*BinaryID);<br>
> return std::move(Readers);<br>
> }<br>
><br>
><br>
> diff --git a/llvm/tools/llvm-cov/CMakeLists.txt b/llvm/tools/llvm-cov/CMakeLists.txt<br>
> index 300bbd0bc6dbd..7acc87e08a9ef 100644<br>
> --- a/llvm/tools/llvm-cov/CMakeLists.txt<br>
> +++ b/llvm/tools/llvm-cov/CMakeLists.txt<br>
> @@ -21,3 +21,5 @@ add_llvm_tool(llvm-cov<br>
> SourceCoverageViewText.cpp<br>
> TestingSupport.cpp<br>
> )<br>
> +<br>
> +target_link_libraries(llvm-cov PRIVATE LLVMDebuginfod)<br>
><br>
> diff --git a/llvm/tools/llvm-cov/CodeCoverage.cpp b/llvm/tools/llvm-cov/CodeCoverage.cpp<br>
> index 2b2eda5d85873..7366059cd242f 100644<br>
> --- a/llvm/tools/llvm-cov/CodeCoverage.cpp<br>
> +++ b/llvm/tools/llvm-cov/CodeCoverage.cpp<br>
> @@ -23,6 +23,10 @@<br>
> #include "llvm/ADT/SmallString.h"<br>
> #include "llvm/ADT/StringRef.h"<br>
> #include "llvm/ADT/Triple.h"<br>
> +#include "llvm/Debuginfod/BuildIDFetcher.h"<br>
> +#include "llvm/Debuginfod/Debuginfod.h"<br>
> +#include "llvm/Debuginfod/HTTPClient.h"<br>
> +#include "llvm/Object/BuildID.h"<br>
> #include "llvm/ProfileData/Coverage/CoverageMapping.h"<br>
> #include "llvm/ProfileData/InstrProfReader.h"<br>
> #include "llvm/Support/CommandLine.h"<br>
> @@ -179,6 +183,8 @@ class CodeCoverageTool {<br>
><br>
> /// Allowlist from -name-allowlist to be used for filtering.<br>
> std::unique_ptr<SpecialCaseList> NameAllowlist;<br>
> +<br>
> + std::unique_ptr<object::BuildIDFetcher> BIDFetcher;<br>
> };<br>
> }<br>
><br>
> @@ -435,7 +441,7 @@ std::unique_ptr<CoverageMapping> CodeCoverageTool::load() {<br>
> ObjectFilename);<br>
> auto CoverageOrErr =<br>
> CoverageMapping::load(ObjectFilenames, PGOFilename, CoverageArches,<br>
> - ViewOpts.CompilationDirectory);<br>
> + ViewOpts.CompilationDirectory, BIDFetcher.get());<br>
> if (Error E = CoverageOrErr.takeError()) {<br>
> error("Failed to load coverage: " + toString(std::move(E)));<br>
> return nullptr;<br>
> @@ -647,6 +653,14 @@ int CodeCoverageTool::run(Command Cmd, int argc, const char **argv) {<br>
> cl::opt<bool> DebugDump("dump", cl::Optional,<br>
> cl::desc("Show internal debug dump"));<br>
><br>
> + cl::list<std::string> DebugFileDirectory(<br>
> + "debug-file-directory",<br>
> + cl::desc("Directories to search for object files by build ID"));<br>
> + cl::opt<bool> Debuginfod(<br>
> + "debuginfod", cl::ZeroOrMore,<br>
> + cl::desc("Use debuginfod to look up object files from profile"),<br>
> + cl::init(canUseDebuginfod()));<br>
> +<br>
> cl::opt<CoverageViewOptions::OutputFormat> Format(<br>
> "format", cl::desc("Output format for line-based coverage reports"),<br>
> cl::values(clEnumValN(CoverageViewOptions::OutputFormat::Text, "text",<br>
> @@ -749,12 +763,18 @@ int CodeCoverageTool::run(Command Cmd, int argc, const char **argv) {<br>
> auto commandLineParser = [&, this](int argc, const char **argv) -> int {<br>
> cl::ParseCommandLineOptions(argc, argv, "LLVM code coverage tool\n");<br>
> ViewOpts.Debug = DebugDump;<br>
> + if (Debuginfod) {<br>
> + HTTPClient::initialize();<br>
> + BIDFetcher = std::make_unique<DebuginfodFetcher>(DebugFileDirectory);<br>
> + } else {<br>
> + BIDFetcher = std::make_unique<object::BuildIDFetcher>(DebugFileDirectory);<br>
> + }<br>
><br>
> if (!CovFilename.empty())<br>
> ObjectFilenames.emplace_back(CovFilename);<br>
> for (const std::string &Filename : CovFilenames)<br>
> ObjectFilenames.emplace_back(Filename);<br>
> - if (ObjectFilenames.empty()) {<br>
> + if (ObjectFilenames.empty() && !Debuginfod && DebugFileDirectory.empty()) {<br>
> errs() << "No filenames specified!\n";<br>
> ::exit(1);<br>
> }<br>
> @@ -867,10 +887,8 @@ int CodeCoverageTool::run(Command Cmd, int argc, const char **argv) {<br>
> }<br>
> CoverageArches.emplace_back(Arch);<br>
> }<br>
> - if (CoverageArches.size() == 1)<br>
> - CoverageArches.insert(CoverageArches.end(), ObjectFilenames.size() - 1,<br>
> - CoverageArches[0]);<br>
> - if (CoverageArches.size() != ObjectFilenames.size()) {<br>
> + if (CoverageArches.size() != 1 &&<br>
> + CoverageArches.size() != ObjectFilenames.size()) {<br>
> error("Number of architectures doesn't match the number of objects");<br>
> return 1;<br>
> }<br>
><br>
> diff --git a/llvm/tools/llvm-objdump/llvm-objdump.cpp b/llvm/tools/llvm-objdump/llvm-objdump.cpp<br>
> index 930b132533cdd..9979a26cf1154 100644<br>
> --- a/llvm/tools/llvm-objdump/llvm-objdump.cpp<br>
> +++ b/llvm/tools/llvm-objdump/llvm-objdump.cpp<br>
> @@ -3198,9 +3198,7 @@ int main(int argc, char **argv) {<br>
><br>
> // Initialize debuginfod.<br>
> const bool ShouldUseDebuginfodByDefault =<br>
> - InputArgs.hasArg(OBJDUMP_build_id) ||<br>
> - (HTTPClient::isAvailable() &&<br>
> - !ExitOnErr(getDefaultDebuginfodUrls()).empty());<br>
> + InputArgs.hasArg(OBJDUMP_build_id) || canUseDebuginfod();<br>
> std::vector<std::string> DebugFileDirectories =<br>
> InputArgs.getAllArgValues(OBJDUMP_debug_file_directory);<br>
> if (InputArgs.hasFlag(OBJDUMP_debuginfod, OBJDUMP_no_debuginfod,<br>
><br>
> diff --git a/llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp b/llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp<br>
> index 1b86134dda516..ed24e85502911 100644<br>
> --- a/llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp<br>
> +++ b/llvm/tools/llvm-symbolizer/llvm-symbolizer.cpp<br>
> @@ -443,13 +443,7 @@ int main(int argc, char **argv) {<br>
><br>
> LLVMSymbolizer Symbolizer(Opts);<br>
><br>
> - // A debuginfod lookup could succeed if a HTTP client is available and at<br>
> - // least one backing URL is configured.<br>
> - bool ShouldUseDebuginfodByDefault =<br>
> - HTTPClient::isAvailable() &&<br>
> - !ExitOnErr(getDefaultDebuginfodUrls()).empty();<br>
> - if (Args.hasFlag(OPT_debuginfod, OPT_no_debuginfod,<br>
> - ShouldUseDebuginfodByDefault))<br>
> + if (Args.hasFlag(OPT_debuginfod, OPT_no_debuginfod, canUseDebuginfod()))<br>
> enableDebuginfod(Symbolizer, Args);<br>
><br>
> if (Args.hasArg(OPT_filter_markup)) {<br>
><br>
><br>
><br>
> _______________________________________________<br>
> llvm-commits mailing list<br>
> <a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a><br>
> <a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><br><div style="line-height:1.5em;padding-top:10px;margin-top:10px;color:rgb(85,85,85);font-family:sans-serif;font-size:small"><span style="border-width:2px 0px 0px;border-style:solid;border-color:rgb(213,15,37);padding-top:2px;margin-top:2px">Daniel </span><span style="border-width:2px 0px 0px;border-style:solid;border-color:rgb(51,105,232);padding-top:2px;margin-top:2px">Thornburgh |</span><span style="border-width:2px 0px 0px;border-style:solid;border-color:rgb(0,153,57);padding-top:2px;margin-top:2px"> <a href="mailto:dthorn@google.com" target="_blank">dthorn@google.com</a></span></div><br></div></div></div></div>