[Lldb-commits] [lldb] Add ability to specify running LLDB API testsuite by json description (PR #89764)

[Alex Langford via lldb-commits]

https://github.com/bulbazord commented:

This change does 2 things, both of which are quite substantial on their own. Can you break them up into two changes to make it easier to review?

Higher level feedback:
Please add some documentation for these new changes. The change to dotest.py is minor but a short line about it somewhere (perhaps in the test suite docs?) would be helpful. As for the JSON Config, there is documentation on common build options where this might be useful.

Some feedback on the individual changes:
1) Changing dotest so that it can accept multiple things for `-E`. You solved this by inserting a special token (`start:`) that tells dotest to interpret it differently. Could you not instead change the action for `-E` to be `append`? That way E is already a list of arguments. I see that `start` is there to prevent an issue when the argument starts with `--` right? Is that an issue if you add quotes? What about if you do something like `-E "--first -Second --third"`? Does that work?

2) It looks like you're expected to pass the JSON config file into CMake which does some processing on it to fill out lit.cfg right? Does this mean I'd have to reconfigure if I change the JSON at all? I think it would be easier if we could instead teach dotest.py to read the JSON configuration. The CMake code would then either accept a user-provided JSON file or build a default one.

https://github.com/llvm/llvm-project/pull/89764