[cfe-dev] Wrapping lit commands for device testing

[George Karpenkov via cfe-dev]

Hi,

Tests in compiler-rt run under both iOS simulators and Android devices, which requires “wrapping” the run line with the “adb” command for Android, and adding a prefix / setting some environment variables for simulators/
In case of Android this is done by changing the compilation command to produce a wrapper which already calls “adb”, and in case of iOS simulators by explicitly adding the “%run” command in front of run lines.
The solution is far from perfect though: many tests (e.g. libFuzzer) extensively rely on using arbitrary shell commands to create and manipulate temporary text files.
Performing those commands in a different environment is done in a very ad-hoc manner: from what I see, for sanitizer tests those temporary files are then usually references in the environment variables present in the run command, which are then parsed to carry those files into the executable environment.
Doing this for libFuzzer would be very difficult and would require a lot of modifications to test.

I would prefer a more transparent approach: if LIT configuration was to provide a hook to perform arbitrary replacements/wrapping on each of the executed lines, the wrapping could be performed without modifying tests at all.

My implementation preference would be adding a method  `def wrap(self, s)` (str -> str) to the format class (lit/formats/*), and then using it (if non-empty) on each line being run.
I think that is preferable to a method on a `config` object, since line transformation is a very heavyweight operation, akin to other methods available on the `format` class (test discovery semantics and test execution semantics).
Would there be any objections against such an implementation plan?

Regards,
George