<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Coming back to this after a little
while, I think I'd like to change my position somewhat.<br>
<br>
Given the feedback on this thread, as well as one other <a
href="http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-September/053540.html">recent
request</a> for the same functionality, I'd really like to see
this added to FileCheck. It doesn't have to be in the exact form
I've proposed but I'd like some solution to handling incidental
variations in order.<br>
<br>
The current proposal*,<br>
- does not (IMO) complicate the implementation of FileCheck much,<br>
- does not change the behavior of existing tests, and <br>
- does not complicate or change the writing of tests if you do not
use the new functionality<br>
<br>
Chandler, do you disagree? Are you still opposed to adding this
feature?<br>
<br>
Thanks,<br>
Matthew C.<br>
<br>
[*]I actually have an alternate solution that I will post soon,
but it doesn't change the assertions that I'm making and is a bit
cleaner from a test writer's perspective.<br>
<br>
<br>
On 9/10/2012 9:27 AM, Matthew Curtis wrote:<br>
</div>
<blockquote cite="mid:504DF8D9.5000309@codeaurora.org" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div class="moz-cite-prefix">On 9/7/2012 4:08 PM, Chandler Carruth
wrote:<br>
</div>
<blockquote
cite="mid:CAGCO0Kg+weEmyi-kQZCS_Yx8CyKeamieJhOSZKKeX6WnxNrbRQ@mail.gmail.com"
type="cite">
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Sep 7, 2012 at 8:20 AM,
Matthew Curtis <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mcurtis@codeaurora.org" target="_blank"
class="cremed">mcurtis@codeaurora.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hello all,<br>
<br>
For the hexagon target, we have a couple of tests that
are failing due to variations in the order of checked
text. In these cases the ordering is not directly
relevant to the functionality being tested.<br>
<br>
For example:<br>
<blockquote><tt>; CHECK: memw(##a)</tt><br>
<tt>; CHECK: memw(##b)</tt><br>
<br>
<tt>%0 = load i32* @a, align 4</tt><br>
<tt>%1 = load i32* @b, align 4</tt><br>
</blockquote>
requires that the compiler emit the memory operations
for 'a' and 'b' in that order, even though the intent of
the test might simply be to ensure that each 'load'
results in a memory read.<br>
<br>
I'd like to teach FileCheck to allow tests to be more
tolerant of these incidental variations.<br>
</div>
</blockquote>
<div><br>
</div>
<div>So, I'm not a huge fan of this, but for a strange
reason.</div>
<div><br>
</div>
<div>Fundamentally, I agree with you that these ordering
dependencies are unfortunate in tests. However, I would
make a few observations:</div>
<div><br>
</div>
<div>1) The order of instructions, unlike some things such
as register allocation's selection of registers, should
ideally not vary much. Currently it does more that we
would like due to the inability to unit test single pieces
of the backend, but I don't think we should make FileCheck
better to cope with that, I think we should fix the
underlying problem.</div>
<div><br>
</div>
<div>2) It is usually fairly obvious when two such checks
don't actually have an ordering constraint that is
important, and where it isn't, a comment makes the intent
clear.</div>
<div><br>
</div>
<div> 3) By checking the ordering, we gain at least some
early detection of non-determinism in the code generator.
I definitely caught several bugs in the block placement
pass while I was working on it due to this very "feature"
of the test suite.</div>
<div><br>
</div>
<div><br>
</div>
<div>Given these points, the value tradeoff here of making
FileCheck *significantly* more complicated (and the tests
significantly more complex as well) versus the time
savings updating test cases when the ordering changes...</div>
<div><br>
</div>
</div>
</div>
</blockquote>
<br>
To be honest, I'm fairly new to LLVM and Clang so I don't have a
good feel for how often this will come up. I've encountered two
tests over the past month and a half that have regressed due to
ordering changes. In one case the order varies based on target, so
I would have to duplicate the test and enable/disable one or the
other based on target. Not great, but if it's just one test
probably acceptable.<br>
<br>
As far as complexity, I didn't feel like the complexity added to
FileCheck was that significant, though relative to how often the
feature would (or should) get used perhaps it is. I was more
concerned about adding 2-3 new directives to the user interface.
In the end, that acceptable given that users would only see the
additional complexity if they needed it.<br>
<br>
If no one else feels like they would benefit much from this, I'll
probably just wait and see how much of a maintenance burden this
really is. If it continues to be an issue I can bring it back up
to the community.<br>
<br>
<blockquote
cite="mid:CAGCO0Kg+weEmyi-kQZCS_Yx8CyKeamieJhOSZKKeX6WnxNrbRQ@mail.gmail.com"
type="cite">
<div class="gmail_extra">
<div class="gmail_quote">
<div>But maybe others who have spent more time hammering
against these problems in the backend have a different
sense of the magnitude of the problem?</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <br>
The attached patch implements one possible solution. It
introduces a position stack and a couple of directives:<br>
<ul>
<li>'CHECK-PUSH:' pushes the current match position
onto the stack.</li>
<li>'CHECK-POP:' pops the top value off of the stack
and uses it to set the current match position.</li>
</ul>
The above test can now be re-written as:<br>
<blockquote><tt>; CHECK-PUSH:</tt><br>
<tt>; CHECK: memw(##a)</tt><br>
<tt>; CHECK-POP:</tt><br>
<tt>; CHECK: memw(##b)</tt><br>
<br>
<tt>%0 = load i32* @a, align 4</tt><br>
<tt>%1 = load i32* @b, align 4</tt><br>
</blockquote>
which handles either ordering of memory reads for 'a'
and 'b'.<br>
<br>
Thoughts?<br>
<br>
Thanks,<br>
Matthew Curtis<span class="HOEnZb"><font color="#888888"><br>
<pre cols="72">--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation</pre>
</font></span></div>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a moz-do-not-send="true"
href="mailto:LLVMdev@cs.uiuc.edu" class="cremed">LLVMdev@cs.uiuc.edu</a>
<a moz-do-not-send="true"
href="http://llvm.cs.uiuc.edu" target="_blank"
class="cremed">http://llvm.cs.uiuc.edu</a><br>
<a moz-do-not-send="true"
href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev"
target="_blank" class="cremed">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
LLVM Developers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a class="moz-txt-link-freetext" href="http://llvm.cs.uiuc.edu">http://llvm.cs.uiuc.edu</a>
<a class="moz-txt-link-freetext" href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation</pre>
</body>
</html>