[cfe-dev] Analysis-based toolchain tools/job actions without output

[Andrzej Warzynski via cfe-dev]

Hi Jonathan,

Sorry for not responding earlier.

What you've described agrees with my limited experience here - the 
compiler driver creates Jobs that correspond to various compilation 
phases. And the output from one phase is fed into the next one. This is 
very nicely illustrated with `-ccc-print-phases`. I've not tried it 
myself, but suspect that diverging from this model is going to be rather 
tricky.

Perhaps you could/should tweak the job that generates the binary? That 
would be the backend job, right? You'd have to abandon the idea of using 
a custom tool and implement this in LLVM. But given how powerful LLVM 
is, I imagine that it wouldn't be too hard.

And if that's not an option, then I guess that probably you want a dummy 
compiler driver phase that doesn't output anything. I don't have much 
experience with the API surrounding phase control in the driver. You 
could start by checking the most recent patches touching e.g.:
   * include/clang/Driver/Types.def
   * include/clang/Driver/Phases.h
   * lib/Driver/Phases.cpp

I hope that this helps at least a tiny bit!

-Andrzej


On 19/02/2021 10:32, Jonathan Smith via cfe-dev wrote:
> Greetings everyone,
> 
> I'm using a slightly modified version of Clang and LLVM to build
> binaries that are effectively firmware for custom hardware -- code
> that gets compiled down to flat executable code which will be dropped
> into some flash memory on a device at a known, fixed address. These
> files exit the compilation process from LLD as ELF executables before
> being validated and converted to binary blobs via objcopy. The
> validation process takes place using a custom tool, and all it does is
> check to make sure all the final ELF sections are read-only. (There
> are probably better ways of doing this, but that's certainly outside
> of the scope of this email!)
> 
> The validation and extraction processes are accomplished using
> individual build steps in a terrible custom build system (in the name
> of OS compatibility despite the fact that we only use this on a single
> platform) which can't be replaced any time soon. There is a TON of
> code involved in these build steps using this horrible internal build
> system and its continued use is equivalent to indentured servitude for
> technical debt. However, we're stuck with it for the foreseeable
> future.
> 
> In an effort to make this easier and to reduce the amount of garbage
> code my team has to deal with in this mockery of a build system, I
> would like to convert the ELF validation process into a custom job
> action to Clang's compilation pipeline -- specifically, actions
> defined by `clang::Driver::BuildActions()`
> (https://github.com/llvm/llvm-project/blob/d12fa33d736d60d419f86b4ec5f3e77e602d4b1e/clang/lib/Driver/Driver.cpp#L3586).
> I've done this before for tools such as `llvm-link` to provide
> automatic bitcode linking, but that tool produces output which can be
> fed back into the compilation pipeline for further actions such as
> code generation.
> 
> The ELF validation tool produces no output -- it simply checks the
> executable and exits with a 0 status if it determines everything is
> good, or it prints relevant error messages and exits with a 1 status.
> This is where I run into problems. In my attempts to make this tool a
> step in Clang's compilation process, I've not found a way to tell
> Clang to simply use the output from the preceding step/job action as
> the output for this tool. As a bandage fix, I've modified the
> validator tool to accept a `-o=<XYZ>` argument and to use that to
> create a symlink of the input file. I can then feed that symlink back
> to Clang. Clang is happy with this, but it's *not* a good solution.
> 
> I see other job actions that don't produce output (such as those for
> handling `-fsyntax-only` and `-verify-pch`, but those also act as
> terminators for the compilation process.
> 
> Is there a way to do what I'm trying to do within Clang without
> halting the pipeline?
> 
> Respectfully,
> Jonathan Smith (jvstech)
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>