[llvm-dev] [lit] RFC: Per test timeout

[Dan Liew via llvm-dev]

Hi,

I've tinkering my patch today and I've placed the new version on
phabricator for review

http://reviews.llvm.org/D14706


>> Okay. I'll bear that in mind. I currently want to distinguish between a
>> TIMEOUT and UNRESOLVED (I'm not sure what other conditions can lead to
>> this result code) so I will keep it for now.
>
>
> The reason not to is that there are lots of out-of-tree scripts that parse
> the output of LIT (which was designed to match DejaGnu's output). It's best
> not to force everyone update their scripts. Therefore tests that run past
> their allotted time should be marked as FAIL.

Okay. I've already put my patch on phabricator and I have not changed
anything with respect to this.

For my particular use case it is not desirable to have FAIL mean
either the execution failed in some way or the test ran out of time.
If we can't reach consensus here it might be necessary to add some
sort of flag (--deja-gnu-result-codes) that is on by default that
causes TIMEOUT to appear as UNRESOLVED but can be disabled.


>>
>>  >
>>  >> * A new command line option --max-individual-test-time
>>  >
>>  >
>>  > I think you should call it `--timeout=`, and then say in the
>> description that it's a per-test timeout.
>>
>> I agree a shorter name would be nicer. I'm worried about it being
>> confused with --max-time though.
>
>
> The --help description should be good enough to distinguish them.  It'll be
> important however to keep their naming consistent across the variables that
> are used to implement this.
>
>>

Again I haven't changed the naming I used in my initial patch.
Although ``--timeout=`` is convenient to use from the command line
(and I'd be willing to change that)
using that name inside lit's code is very ambiguous so I would not
want to use that name inside the LitConfig object.

Thanks,
Dan.