[PATCH] D82754: [lit] Prevent hang when lit sees non-ASCII characters

[Richard Barton via Phabricator via llvm-commits]

richard.barton.arm added a comment.

Hi @jdenny

Agreed on adding a comment and test - the original patch was a bit information-light, sorry about that.

I am really struggling to figure out how to add a test for this and wonder if you can help/advise.

My environment is python 3.5, ubuntu 16.04 LTS, bash shell with this locale:

> LANG=en_US.UTF-8
>  LANGUAGE=en_US:
>  LC_CTYPE="C"
>  LC_NUMERIC="C"
>  LC_TIME="C"
>  LC_COLLATE="C"
>  LC_MONETARY="C"
>  LC_MESSAGES="C"
>  LC_PAPER="C"
>  LC_NAME="C"
>  LC_ADDRESS="C"
>  LC_TELEPHONE="C"
>  LC_MEASUREMENT="C"
>  LC_IDENTIFICATION="C"
>  LC_ALL=C

In my environment, the test added under D69207 <https://reviews.llvm.org/D69207> - shtest-shell/stdout-encoding.txt - hits the UnicodeDecodeError when I run it directly with `llvm-lit -a`. Setting LC_ALL=en_US.utf-8 avoids the error, obviously. When running the test via make with `LIT_OPTS="-a --filter=shtest-shell" make check-lit` it does not fail, which is, I assume, why the problem has gone un-detected. I cannot work out how to build a test that runs the 'inner lit' in such an environment where the error would be raised. I can make a test that is adapted from shtest-shell/stdout-encoding.txt with this RUN line

  # RUN: not env PYTHONIOENCODING=ascii %{lit} -j 1 -v %{inputs}/shtest-shell-ascii > %t.out

which will trigger the error for me when run with llvm-lit without -a (although the failure mode is to hang, so pretty nasty) but this test passes in make check-all.

I can continue to look further but can you see a trick I am missing? Alternatively, am I on a fool's errand here and a new test will not be possible or practical?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82754/new/

https://reviews.llvm.org/D82754