[cfe-dev] SBCC and emitEmptyMapping

[Max Moroz via cfe-dev]

Yes, "-mllvm -limited-coverage-experimental=true" did help us with
Chromium. Things still are crazy large, but smaller than before.

Also, make sure to use "-sparse" option of llvm-profdata tool. It doesn't
emit function records with 0 execution count.

For instance, when generating code coverage for Chromium, we generate 37 GB
of .profraw files (1000+ dumps from ~370 binaries), but the resulting
.profdata is only ~605 MB. If we merge it without -sparse, it becomes ~878
MB.

On Thu, May 10, 2018 at 9:57 PM Vedant Kumar <vsk at apple.com> wrote:

> I should have mentioned that there is a bug report about the scaling issue:
> https://bugs.llvm.org/show_bug.cgi?id=34533
>
> One promising solution is to give the empty coverage mappings names, mark
> them as linkonce_odr, and store them into a different section. The premise
> of this approach is that clang will emit identical empty mappings for a
> function many times, say, if it's an inline function in a header. By
> marking each of these mappings as linkonce_odr, the linker can unique them
> in the final binary and save a fair amount of space (O(#functions), instead
> of O(#functions * #translation-units)).
>
> As I've switched to a different team at work, I don't have the time to
> work on code coverage bugs at the moment. That said I'd be happy to chat
> about this further or review patches.
>
> vedant
>
> On May 10, 2018, at 9:39 PM, Vedant Kumar <vsk at apple.com> wrote:
>
> + Max
>
> On May 10, 2018, at 1:25 PM, Harp, Thom via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
> Compared to LLVM’s gcov-compatible coverage method, SBCC’s resource
> requirements are much bigger.  The total size of all .profraw files from my
> system is almost 1GB while the same system,
>
>
> Have you tried enabling run-time raw profile merging with "%Nm" (
> https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#running-the-instrumented-program
> )?
>
>
> built for gcov, totals only 150MB of .gcda files.  I removed
> emitEmptyMapping from CoverageMapping.cpp to see what happens,
>
>
> Some engineers working on Chromium tried something similar using an
> experimental cl::opt (-limited-coverage-experimental) which may have a
> comparable same effect. I've CC'd Max who may be able to say more about
> this.
>
>
> Er, s/comparable same/comparable/
>
>
>
> and the SBCC coverage dumps dropped from 1GB to about 26MB.  There are
> also significant gains in runtime memory use and on-disk image size.
>
> I don’t mind losing the zero counts for code that is unused
>
>
> In this case, I'd suggest trying the cl::opt I mentioned above, with the
> caveat that it can skew coverage reporting quite a bit.
>
> IMO having some mapping information for unused code is arguably one of the
> more important use cases for coverage reporting.
>
>
> but I want to understand how clang determines what’s used and what isn’t.
>
>
> See AddDeferredUnusedCoverageMapping, ClearUnusedCoverageMapping,
> and EmitDeferredUnusedCoverageMappings. I'm not the original author so I
> might not have this totally right, but I think the idea here is that:
>
> - In CodeGenModule::EmitTopLevelDecl, we might not be sure a Decl actually
> needs to be emitted, so we mark it as deferred for coverage reporting
> purposes. If we never generate code for it, we'd emit a simple empty
> coverage mapping.
>
> - If it turns out that a Decl marked as deferred is actually emitted, we
> clear the deferred bit, because we no longer want an empty mapping.
>
> best,
> vedant
>
>
>   I traced the decision up to CodeGenPGO::assignRegionCounters in
> CodeGenPGO.cpp, where implicit functions and some constructor/destructor
> variants are eliminated but I can’t explain the bulk of it.
>
> Can someone help me understand how clang determines what code is used and
> what isn’t in this context?
>
> Thanks.
> -thom
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180511/2e3ac601/attachment.html>