<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/69221>69221</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            Hardware ASAN deep-recursion.c test is flaky on AArch64 Linux
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            compiler-rt:hwasan
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          DavidSpickett
      </td>
    </tr>
</table>

<pre>
    As seen in:
https://lab.llvm.org/buildbot/#/builders/184/builds/8047
https://lab.llvm.org/buildbot/#/builders/184/builds/7993

This test case sometimes fails with:
```
+ env HWASAN_OPTIONS=disable_allocator_tagging=1:random_tags=0:fail_without_syscall_abi=0:stack_history_size=5 not /home/tcwg-buildbot/worker/clang-aarch64-sve-vls/stage1/runtimes/runtimes-bins/compiler-rt/test/hwasan/AARCH64/TestCases/Output/deep-recursion.c.tmp
+ FileCheck /home/tcwg-buildbot/worker/clang-aarch64-sve-vls/llvm/compiler-rt/test/hwasan/TestCases/deep-recursion.c --check-prefix=D5
FileCheck error: '<stdin>' is empty.
FileCheck command line:  FileCheck /home/tcwg-buildbot/worker/clang-aarch64-sve-vls/llvm/compiler-rt/test/hwasan/TestCases/deep-recursion.c --check-prefix=D5
```

This bot runs on a Graviton 3 instance, I reproduced it on an Ampere server by doing the following:
```
$ ./bin/clang -fsanitize=hwaddress -O1 ../llvm-project/compiler-rt/test/hwasan/TestCases/deep-recursion.c -o /tmp/test.o -mllvm -hwasan-record-stack-history=libcall
```
Then running this shell script:
```
num=0
while [ true ]
do
 /tmp/test.o
  retcode=$?
  num=$(($num + 1))
  if [[ $retcode == '0' ]]
  then
    echo "Found bug with retcode $retcode on run $num"
 exit 1
  fi
done
```
Results:
```
Found bug with retcode 0 on run 946
Found bug with retcode 0 on run 163
Found bug with retcode 0 on run 11
Found bug with retcode 0 on run 224
Found bug with retcode 0 on run 10
Found bug with retcode 0 on run 469
Found bug with retcode 0 on run 1095
```
>From what I gather, the fault should *always* be found. These results also prove that it isn't some sort of pipefail issue in the RUN line.

I tried removing `-hwasan-record-stack-history=libcall` and:
```
Found bug with retcode 0 on run 47
Found bug with retcode 0 on run 128
Found bug with retcode 0 on run 197
```
Clearly that's not enough to say if one is failing to find it more frequently, but you get the idea. Something is going on here.
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzMVk9v47YT_TT0ZSBBomQ5OvigxD__NkCxKTYpejQocSyxoUiVf-x1P31Byt6kaYI1uj0UEGyRGj7OPPLNDLNW9ApxTZa3ZLlZMO8GbdYbdhD8cRLdMzq3aDU_rRsLFlGBUKRoSLYhWTM4N9kwoltCt5K1qZSHMdWmJ3TbeiF5q138WFwm0FhCt_lNeZkIw5usXP27iKu6LmbE-fdpEBYcWgcdswhWj-jEiBb2TEgLR-GGb1GRKjs_85DeAqoDfPq1eWw-7x5-frp_-PxIig0XlrUSd0xK3TGnzc6xvheqJ8UmJ0VjmOJ6DJOWFJuMFE3YbBf20t7t7Ml2TModa8X5s3Wse94NwjptTjsr_kBSbJagtANCt4MekdCt64598oqJozbPaAjddpKpPmHMdENVJvaAyUEGKqxjPeaEbo1XMeZXr0krVBh3epyERJOYABl4ChsemWWK0G3TfLn7VAV-n9C6O2YjxoN3kw92HHFKDHbeWKFV2qVunF6o2wqJdwN2zz8SQ7gE33XztXNvfYIk6YITyWRwL76SYrNZzj6--IfGaEOKBghdkeLOOh6u-v8IXYGwgOPkTunbNZ0eR6Y4SKEwLP3Ph_v2cr-oo9UOjFcWtAIG_zfsIJxWUIBQ1jHVIaF3cA8GJ6O575CDcNFWQTNOaBAsmgMaaE_AtVA9uAFhr6XUxyiKj-RVQhqkK9SFEUj2linhZgEMR8a5QWshecghTc_8JJPRv2HnfpAnHc4pXNh5YaohGQM8JDNAsNaGJ1GbyVmbpNhI0Qb1vhvS04AqUKlmEoQFO6CUYDsjJvcREcqPMQ_EwXEQEoEsb8EZH1428zzX8_9br8-zYNB1mgfaCC1Jsb3Mz-Bhjt7Ep1R-hKDPnNA6PGdDsYdYCW6B0PKMBmFpsQm6yIIagjcXhyCcsrq8A2A3BErpVnvFofV9zK3wDekFVEeOYHaFUHrGwK_CQX4B3ItL3ArfZe0LWi-d_YjUD9zILrvXZXWdYV4VVxrm19lRWl4JeGUoZVVfC1i_nwy2Ro9wHJiDe-iZG0KOuptVzLx0YAftJQdCGyaP7GQJbaANEveKp_A0oEUw83kAk1bDZPQBwQVE4UBYRejKxeILVhsHeg-TmDCURRDWegSh4n5ffvkcU2r6OkvdgzMCORgc9SFIi1TZlSqtMmCK_9NLcmlOvkssvbnSsF6968idRGbkKTJG6MrG0o9K-34Ap8GyU9CnVhgKUmAtJhgNe6FiNh61Qdgb_N2jcvIUTq_1Dk7aQ48uMis4shQeQ_8zhNXCQh-TtVYwoMF0wdcFr4uaLXCdV_WqrOo6zxfDOq-wvMnKqkOes2W7arO8KpbtipfdipareiHWNKNFnuVVvqTLbJVWJe73Fb3hZVUynlekzHBkQn7r6Rbx0NdVTWm-kKxFaWMfSunrlF40l1xOQ4dq1jH9t763pMyksM6-ADrhJK4_McOPzCCEjg3-lvRjHxgYlOz5FAJvmliB4Seh_NeFN3L91z60F27wbdrp8aU4v6lBMZJQYmIwfwYAAP__PIyXNg">