[llvm-dev] Using source-based code coverage on baremetal

[Peter Smith via llvm-dev]

Hello Eli,

I think that this would be very useful for bare-metal systems,
particularly for those without trace capture or the software to
analyze it. I seem to remember an internal proof of concept experiment
to see if we could get profiling working in an embedded toolchain,
which we did after making similar, but toolchain specific,
linker/library modifications that you did.

I think the most valuable contribution would be in example
documentation such as the steps you outline above. Moreover I think
that more documentation and recipes for building and testing
compiler-rt for bare-metal would be extremely valuable.

I'm not much of an expert on profiling, or compiler-rt but I'm willing
to help out where I can.

My understanding is that COMPILER_RT_BAREMETAL_BUILD is indeed used to
exclude objects from the library that would be incompatible with a
bare-metal target. It looks like it was introduced in
https://reviews.llvm.org/D33018.

I've not looked into how this works in compiler-rt/compiler right now
so apologies if this is somewhat speculative.
As I understand it, the section __start and __end symbols is a GNU ld
convention. I think one way of baking this in would be to separate out
functions that access the __start and __end symbols into
getSectionStart(), getSectionEnd() functions (probably weak
definitions) that are in their own .o file. If a bare-metal toolchain
has a different set of linker magic/convention it can implement its
own getSectionStart() and getSectionEnd(). On the assumption that most
bare-metal toolchains have their own target and driver a flag sounds
like a good idea.

Peter


On 6 September 2017 at 02:55, Friedman, Eli via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> Hi all,
>
> I think using code coverage on baremetal has come up once or twice on
> llvmdev, but I don't think anyone has actually written up how the workflow
> works, or what issues come up.  This description is based on work done
> together with my colleague Weiming Zhao.
>
> By "baremetal" here, I mean an embedded environment without an operating
> system.  We specifically used a ARM target with semihosting, but similar
> steps should work elsewhere.
>
> The workflow:
>
> 1. First, you need a copy of libclangrt_profile.a, stripped down for the
> baremetal target.  (More on this below.)
>
> 2. Then, you need to change the source code to call into it; since a
> baremetal image doesn't exit like an operating system process, you need to
> insert code somewhere to write out the profile data yourself.  We used
> __llvm_profile_get_size_for_buffer() and __llvm_profile_write_buf() for this
> (and semihosting APIs to transfer the resulting buffer to the host).
>
> 3. Then, you have to edit your linker script to include the necessary
> sections.  __llvm_prf_names, __llvm_prf_data, and __llvm_prf_cnts need to be
> included in the final image.  And __llvm_covmap needs to be in a
> non-allocatable section in the ELF output.  And depending on how your linker
> behaves, you might need to explicitly mark all of these sections KEEP so
> parts don't get dropped.  This is actually the trickiest part, in the sense
> that messing it up lead to obscure issues which are difficult to debug. At
> best, you get an error message like "No coverage data found" or "Malformed
> instrumentation profile data".  Or, if you're using a build of LLVM more
> than a few months old, coverage data can be silently dropped.
>
> 4. Then, you add "-fprofile-instr-generate -fcoverage-mapping -mllvm
> -enable-value-profiling=false" to your CFLAGS.
>
> 5. Then, you build and run the image in the semihosted environment. If
> everything works, you get a file with raw profile data, and use the normal
> workflow to convert it into a report.
>
> Areas that required LLVM changes:
>
> 1. The copy of libclangrt_profile.a for the target.  Given that we already
> were using builtins from compiler-rt, the primary changes required are
> enabling the profile library and excluding a bunch of files from the build
> (since baremetal doesn't have a filesystem, system calls, etc.).  I'll look
> into posting patches when I have time, but it might take me a little while
> for me to figure out how to cleanly modify the build, and verify everything
> actually works on trunk.  It looks like there's a CMake variable
> COMPILER_RT_BAREMETAL_BUILD which is supposed to be turned on for this sort
> of environment?
>
> 2. Changing the compiler and compiler-rt to use __start and __end symbols to
> find the sections, rather than .init code.  This isn't strictly necessary,
> but our linker supports __start and __end, and this was easier than changing
> the baremetal image to handle a .init section.  See
> needsRuntimeRegistrationOfSectionRange in
> lib/Transforms/Instrumentation/InstrProfiling.cpp; we currently only
> whitelist a few platforms.  Not sure what would be appropriate here; maybe
> we could assume any *-none-* triple supports __start and __end symbols?  Or
> maybe control it with a flag somehow? Or something else I'm not thinking of?
>
> Other problem areas:
>
> 1. We turned value profiling off because we were running into runtime
> issues; specifically, we had infinite recursion because we instrumented
> malloc.  It isn't really important to have in this context (coverage reports
> currently don't use the data at all), but is there some way to improve this
> workflow to involve fewer magic command-line flags?
>
> 2. The error messages produced by llvm-profdata and llvm-cov tools could
> probably be improved, to describe the actual issue in more detail.
>
> Next steps:
>
> The next steps here depend on community interest, I guess... has anyone else
> tried something like this?  Is anyone interested in my patches?  Should we
> add a section to the coverage documentation?
>
> -Eli
>
> --
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux
> Foundation Collaborative Project
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev