<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Feb 29, 2016, at 11:28 PM, Chandler Carruth <<a href="mailto:chandlerc@google.com" class="">chandlerc@google.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">I really don't understand this. Why is a C++ reference useful? <span style="line-height:1.5" class="">Instead, we have an IR reference: the unoptimized IR.</span></div></div></blockquote><div><br class=""></div><div>Mmmmm, thinking more about it, yes we could compile the IR two times and use the unoptimized one as a reference.</div><div>However, right now the tests are only running llc, so the optimization level is not that impactful and it is not clear to me that a miscompile would be detected this way.</div><div><br class=""></div><div>Which lead me to a separate question for Alina: is this IR "unoptimized"? What kind of optimization pipeline is used by Halide? Is the plan to only exercise the CodeGen?</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">I think it pretty important to not phrase everything in terms of C or C++ or Clang... =/</div><div class=""><br class=""></div><div class="">What if there are IR constructs that simply cannot be produced by C++? We're adding lots of those for managed languages.</div></div></div></blockquote><div><br class=""></div><div>Having a first-class test-suite for IR with a managed runtime is probably a whole new discussion :)</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">If there is a practical concern with having IR-based test cases, I'd like to actually discuss those head on rather than fishing for phrasing even more of our testing in terms of a single language and front-end.</div></div></div></blockquote><div><br class=""></div><div>I think we're not talking about the same thing, because I feel that "phrasing the test in term of a single language" is misrepresenting my point about having a reference in C++. I mentioned in a previous email that I'm all for having the Halide source file along the IR to "phrase the test".</div><div>The point about the reference is "how to validate the output of running the IR test"?</div><div>I'm fine with alternate design, like having the reference computed in Python or Julia (or ...), but I assumed C++ would seem just more natural to everyone and integrates with the test infrastructure (written in C++).</div><div>I'm also fine with having a textual reference output file instead of a C++ code that computes it (I believe this is what we are doing for the C/C++ tests in the test-suite), but we are losing the current scheme to have a random input for these tests.</div><div>(by the way Alina: the seed needs to be kept and printed on errors for reproducibility, also a command line option would be nice to pass it).</div><div><br class=""></div><div>-- </div><div>Mehdi</div><div><br class=""></div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Tue, Mar 1, 2016 at 2:16 AM Mehdi Amini via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><blockquote type="cite" class=""><div class="">On Feb 29, 2016, at 10:50 PM, Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank" class="">hfinkel@anl.gov</a>> wrote:</div><br class=""><div class=""><div style="font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-family:arial,helvetica,sans-serif;font-size:10pt" class=""><br class=""><br class=""><hr class=""><blockquote style="border-left-width:2px;border-left-style:solid;border-left-color:rgb(16,16,255);margin-left:5px;padding-left:5px;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt" class=""><b class="">From:<span class=""> </span></b>"Mehdi Amini via llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>><br class=""><b class="">To:<span class=""> </span></b>"Alina Sbirlea" <<a href="mailto:alina.sbirlea@gmail.com" target="_blank" class="">alina.sbirlea@gmail.com</a>><br class=""><b class="">Cc:<span class=""> </span></b>"llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>><br class=""><b class="">Sent:<span class=""> </span></b>Monday, February 29, 2016 7:06:51 PM<br class=""><b class="">Subject:<span class=""> </span></b>Re: [llvm-dev] RFC: Add bitcode tests to test-suite<br class=""><br class=""><div class=""><br class=""><br class="">Sent from my iPhone</div><div class=""><br class="">On Feb 29, 2016, at 3:39 PM, Alina Sbirlea <<a href="mailto:alina.sbirlea@gmail.com" target="_blank" class="">alina.sbirlea@gmail.com</a>> wrote:<br class=""><br class=""></div><blockquote class=""><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Feb 29, 2016 at 2:06 PM, Mehdi Amini<span class=""> </span><span dir="ltr" class=""><<a href="mailto:mehdi.amini@apple.com" target="_blank" class="">mehdi.amini@apple.com</a>></span><span class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><div class=""><div class=""><blockquote class=""><div class="">On Feb 29, 2016, at 1:50 PM, Alina Sbirlea <<a href="mailto:alina.sbirlea@gmail.com" target="_blank" class="">alina.sbirlea@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Feb 29, 2016 at 12:18 PM, Mehdi Amini<span class=""> </span><span dir="ltr" class=""><<a href="mailto:mehdi.amini@apple.com" target="_blank" class="">mehdi.amini@apple.com</a>></span><span class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote class=""><div class="">On Feb 29, 2016, at 11:40 AM, Mehdi Amini via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class=""><div class=""><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote class=""><div class=""><br class="">On Feb 29, 2016, at 11:16 AM, Alina Sbirlea via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class="">All,<div class=""><br class=""></div><div class="">To get the discussion going in a focused manner, here is an initial patch with a running test. The test is from the Halide suite and is checking the correctness of several simd operations.</div><div class="">(Notes: the patch is large due to the number of operations being tested; </div><div class="">I expect a lot of changes before actually landing it, this is simply to continue the discussion using a concrete example.)<br class=""></div><div class=""><a href="http://reviews.llvm.org/D17726" target="_blank" class="">http://reviews.llvm.org/D17726</a><br class=""></div></div></div></blockquote><div class=""><br class=""></div><div class="">I can't figure how to download the patch *with the bitcode files* from Phabricator. Can you push this on github (or somewhere else)? (or if I missed how to proceed...).</div></div></div></blockquote><div class=""><br class=""></div></span>I was able to figure how get them "one by one", it would still be more convenient to have an archive or a repo to clone somewhere.</div><div class=""><span class=""><br class=""><blockquote class=""><div class=""><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote class=""><div class=""><div dir="ltr" class=""><div class="">A few questions/todos to start the discussion:</div><div class="">1. What is a good location for these tests? They are in a separate Bitcode directory atm, but using the llvm_multisource. This may change to more closely model the approach for external tests (see next item). </div></div></div></blockquote></div></div></blockquote><div class=""><br class=""></div></span><div class="">A good location would be their own external repository IMO :)</div><span class=""><div class=""><br class=""></div><blockquote class=""><div class=""><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote class=""><div class=""><div dir="ltr" class=""><div class="">2. There is a single .cpp file testing all operations provided by individual bitcode files. I expect this to change. Instead of using llvm_multisource to have the same test run with specific arguments, each run testing a single operation.</div><div class="">3. The building approach I took is to first link all bitcode files into a single one, then obtain the assembly for it, which cmake knows to take as an input source. </div></div></div></blockquote></div></div></blockquote><div class=""><br class=""></div></span><div class="">Yeah, so I'd rather have a split-build model, with a split execution model. Having a gigantic bitcode file to debug an issue is not friendly.</div><div class="">I'd expect to have a .cpp file that contains the main and the logic to run test, and then every test that is linked-in to be executed, a bit like gtests is doing (there are multiple registering mechanisms that would avoid to declare explicitly a test in the header).</div><div class="">-> filters.h and filter_headers.h should just go away.</div></div></div></blockquote><div class=""><br class=""></div><div class="">I agree, this is related to point 2. The plan here is to update the current test .cpp file to test each operation individually. In this model it will be enough to link with a single bitcode file per test.</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class=""><br class=""></div><div class="">Also on the test in general: we should have an idea for each test what it is doing and how.</div><div class="">I was expecting your tests to be on the pattern of having an implementation in C++ and an implementation in Halide bitcode of a filters (or whatever) and run both on random data and verifies that the result is matching.</div><div class="">Unfortunately from what I can see you are feeding the tests with random data, and the tests are "blackboxes" that set an error flag if they detect an error. </div><div class="">This is not super robust: the compiler can mess with the error checking and eliminate it for instance, making any error undetected. </div></div></div></blockquote><div class=""><br class=""></div><div class="">The Halide bitcode filters compare the result of vectorized operations vs scalar runs of the same code. The error code against which we compare the output will be set to loose tolerance - it is currently 0. We're interested in codegen bugs that return the wrong value entirely, not accuracy differences (especially for floating point tests).</div><div class="">With the new error threshold, the data fed into may be random or read from provided input files, I can do either.</div><div class="">The filters will still look somewhat like blackboxes, though the name of the filter says what operation it's being tested and the disassembled bitcode files are reasonably readable.</div><div class="">Using your suggestion, the driver .cpp file will test one operation at a time (argvs set accordingly) and return right away once an error is found. Sound about right?</div></div></div></div></div></blockquote><div class=""><br class=""></div></div></div><div class="">All of this is great. </div><div class="">The part that is not clear to me is why isn't it to have (what does it buy us over, or why is it better for us compared to) a possible a C/C++ reference implementation of the filter, and hoisting (and refactor) all the logic to feed the tests and validate the output *out* of the filters. A filter would be just the mathematical function performed and nothing more (separation of concerns, more robust framework, easier debugging when things-go-wrong, etc.).</div></div></div></blockquote><div class=""><br class=""></div><div class="">I believe the answer is that Halide generates vectorized code in a way that is not generated by llvm when starting from C/C++. </div></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">I don't really see how *this* addresses my point. This is justifying why your bitcode is interesting and why we are having this conversation at all :)</div><div class="">It does not say why we can't have a scalar naive C/C++ impl along with the bitcode for filter.</div><div class=""><br class=""></div><br class=""><blockquote class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">Having a C/C++ scalar reference would involve quite a bit of effort for all tests. The primary reason Halide is being used is that you don't need to write a lot of C/C++ code to get different optimizations for the same code (e.g. vectorized vs scalar is a one line difference).<span class=""> </span></div></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Yes, this is what is nice with Halide: "write once, codegen multiple variant". But it does not mean you can't write a c++ reference for every Halide filter (not for every codegen variant of a filter!)</div><div class=""><br class=""></div><div class="">It's been 2 years since my last experiments with Halide, but my memories were that there was a C backend?</div><div class="">I had in mind for each test to have (possibly in a separate directory for each test):</div><div class=""> - the halide source for the filter.</div><div class=""> - the c/c++ (maybe generated?) for the filter.</div><div class=""> - the bitcode generated for the filter (potentially multiple variant depending on the CPU support and/or the schedule).</div><div class=""><br class=""></div><div class="">Then some common code/infrastructure to interact with the individual filters, loading them, executing the variants for a filter, and checking results.</div><div class=""><br class=""></div><div class="">If the reference c/c++ can't be generated by halide (or obtained somehow), and we can't do better than the current tests infrastructure you have, then I'm worried about the cost/benefit for this test-suite.</div><div class=""><br class=""></div></blockquote><br class="">I think that a C/C++ version would be nice to have, but not necessary.</div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-family:arial,helvetica,sans-serif;font-size:10pt" class="">IR generated by non-Clang frontends and/or IR going through alternate optimization pipelines tend to hit bugs that are much harder to hit with Clang alone.</div></div></blockquote><div class=""><br class=""></div></div></div><div style="word-wrap:break-word" class=""><div class=""><div class="">To make it clear: the point of the C++ reference I was asking for is *not to stress clang* at all. It is intended to compute the *golden result* to be compared to the runs of each variant for a filter. Having a reference is important when a diff is detected in the output and you need to figure out what is going on.</div></div></div><div style="word-wrap:break-word" class=""><div class=""><div class=""><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-family:arial,helvetica,sans-serif;font-size:10pt" class=""> It would help to have a description of what each test does, but including, for example, the Halide source code for each test will hopefully be enough of a guide.<br class=""><blockquote style="border-left-width:2px;border-left-style:solid;border-left-color:rgb(16,16,255);margin-left:5px;padding-left:5px;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt" class=""><div class=""></div><br class=""><blockquote class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">So we can get really fast test coverage for possible codegen bugs by comparing that different layout optimizations in Halide give the same result. </div><div class="">I think having each filter tested separately should give a good separation of concerns and easy debugging for each particular test.</div></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">This is great for halide validation, we are all agreeing with this I think. The question is where is the tradeoff for the LLVM project. I'm trying to make sure that the extra coverage doesn't come with a burden to debug and triage issues when something will break: i.e. the tests need to be very friendly to interact with.<span class=""> </span></div></blockquote><br class="">I don't think that any non-trivial tests are truly "friendly" to interact with.</div></div></blockquote><div class=""><br class=""></div></div></div><div style="word-wrap:break-word" class=""><div class=""><div class="">Yeah, I just think we shouldn't make it arbitrarily worse by not having a good infrastructure to begin with :)</div><div class=""><br class=""></div><div class="">In the end, if we expect this suite to be accepted and maintained as a "first-class citizen" by LLVM developers (i.e. accepting things like reverting a commit if it breaks something in this suite), we'd better make sure the burden to interact with it is minimal.</div></div></div><div style="word-wrap:break-word" class=""><div class=""><div class=""><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-family:arial,helvetica,sans-serif;font-size:10pt" class=""> Tracking down self-hosting bugs is not friendly, and those aren't anywhere near the worst ;) --</div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-family:arial,helvetica,sans-serif;font-size:10pt" class="">These tests with their simple driver seem like good input that bugpoint can reduce (assuming the tests runtimes are not too long), and that's friendlier than most of the other multisource tests.<br class=""></div></div></blockquote><div class=""><br class=""></div></div></div><div style="word-wrap:break-word" class=""><div class=""><div class="">If we're talking exclusively about crashes, I agree. However if we're considering correctness issue as well (miscompiles), I believe that the structural changes I proposed are very important to easily perform bugpoint on them for instance (or bugpoint would just turn the test into "return false;").</div><div class="">Also I believe these changes are necessary to perform timing measurement for these tests, if we are interested in the quality of the optimization/codegen (to be hooked into something like <a href="https://github.com/google/benchmark" target="_blank" class="">https://github.com/google/benchmark</a> ?).</div><div class=""><br class=""></div><div class="">Best,</div></div></div><div style="word-wrap:break-word" class=""><div class=""><div class=""><br class=""></div><div class="">-- </div><div class="">Mehdi</div></div></div><div style="word-wrap:break-word" class=""><div class=""><div class=""><br class=""></div><div class=""><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-family:arial,helvetica,sans-serif;font-size:10pt" class=""><br class=""> -Hal<br class=""><br class=""><blockquote style="border-left-width:2px;border-left-style:solid;border-left-color:rgb(16,16,255);margin-left:5px;padding-left:5px;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt" class=""><div class="">This is the motivation for my comments so far.</div><div class="">Other people in the community may have a different opinion/appreciation of the situation, this just represents my current thoughts.</div><div class=""><br class=""></div><div class="">Hope it makes sense.</div><div class=""><br class=""></div><div class="">-- </div><div class="">Mehdi</div><div class=""><br class=""></div><br class=""><blockquote class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><span class=""><div class=""><br class=""></div><blockquote class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="">Also, just looking quickly at one IR I'm surprised by things like:</div><div class=""><br class=""></div><div class="">"assert succeeded165": ; preds = %"assert succeeded146"<br class=""> %buf_host181 = getelementptr inbounds %struct.buffer_t, %struct.buffer_t* %error_op_pcmpeqq_272.buffer, i64 0, i32 1<br class=""> %23 = bitcast i8** %buf_host181 to double**<br class=""> %error_op_pcmpeqq_272.host226227232 = load double*, double** %23, align 8<br class=""> %24 = icmp eq %struct.buffer_t* %error_op_pcmpeqq_272.buffer, null<br class=""> br i1 %24, label %"assert failed183", label %"assert succeeded184", !prof !4<br class=""><br class=""></div><div class="">Here you have as check for nullptr at %24, but you already loaded %error_op_pcmpeqq_272.host226227232 from this pointer just before!</div></div></div></blockquote><div class=""><br class=""></div><div class="">It's checking that the host value loaded from buffer_t is not null. I don't see what's wrong with this. What am I missing?</div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">I may be misreading it, my impression when skimming through the code was that it seems equivalent to:</div><div class=""><br class=""></div><div class="">foo(buffer_t *out) {</div><div class=""> auto value = out->host;</div><div class=""> if (!out) {</div><div class=""> error("nullptr");</div><div class=""> }</div><div class="">}</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">In case I haven't been clear: I think this work is valuable for the project, and thank you for putting some effort into it :)</div><span class=""><font color="#888888" class=""><div class=""><br class=""></div><div class="">-- </div><div class="">Mehdi</div></font></span><div class=""><div class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><br class=""><blockquote class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><span class=""><div class=""><br class=""></div><div class=""><br class=""></div><blockquote class=""><div class=""><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote class=""><div class=""><div dir="ltr" class=""><div class="">A separate discussion is on reading metadata (mcpu and mattr) in llc. I added a script to work around that for now.</div></div></div></blockquote></div></div></blockquote><div class=""><br class=""></div></span><div class="">The generic way of doing it in llvm is (I think) to use function attributes:</div><div class=""><br class=""></div><div class="">attributes #0 = { "target-cpu"="x86-64" "target-features"="+avx2" }<br class=""><br class=""></div><div class="">You shouldn't need it on the command line I think?</div></div></div></blockquote><div class=""><br class=""></div><div class="">Yes, I believe so too. Currently these are set in mcpu and mattr by Halide and not read in by llc, hence the need for feeding them as parameters. It's a separate issue that we'll need to go into in depth, but I don't want it to interfere with getting feedback on how to best publish these tests.</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><span class=""><font color="#888888" class=""><div class=""><br class=""></div><div class="">-- </div><div class="">Mehdi</div></font></span><span class=""><div class=""><br class=""></div><div class=""><br class=""></div><br class=""><blockquote class=""><div class=""><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">Looking forward to your feedback!</div><div class=""><br class=""></div><div class="">Thanks,<br class=""></div><div class="">Alina</div><div class=""><br class=""></div><div class=""><br class=""></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Fri, Feb 19, 2016 at 6:50 AM, Kristof Beyls<span class=""> </span><span dir="ltr" class=""><<a href="mailto:kristof.beyls@arm.com" target="_blank" class="">kristof.beyls@arm.com</a>></span><span class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class=""><span class=""><br class=""><br class=""><div class="">On 18/02/2016 19:12, Alina Sbirlea via llvm-dev wrote:<br class=""></div><blockquote class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br class="">I have more questions for Alina. What kind of tests do you have:<br class=""><br class="">- "the compiler takes the bitcode and generates code without crashing"<br class="">- "the compiled test runs without crashing"<br class="">- "the compiled test will produce an output that be checked against a reference"<br class="">- "the compiled test is meaningful as a benchmarks"<br class=""></blockquote><div class=""><br class=""></div><div class="">We have all 4 kinds of tests in Halide. The bitcode files for the first category is already available and I'm working on building the ones for the next 3. We'd like to include all incrementally.</div><div class=""> </div><br class=""></div></div></div></blockquote></span>It seems to me that the first category ("the compiler takes the bitcode and generates code without crashing") are tests that should be added to the "make check-all" tests in the LLVM subproject, rather than the test-suite subproject?<br class="">Or if these tests currently don't crash the compiler anymore, the bugs must have been fixed, and there should already be equivalent tests?<br class=""></div></blockquote></div><br class=""></div>_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class=""></div></blockquote></div><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">_______________________________________________</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">LLVM Developers mailing list</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">llvm-dev@lists.llvm.org</a><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></span></div><br class=""></div></blockquote></div><br class=""></div></div></div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div></div></blockquote><br class="">_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class=""></blockquote><br class=""><br class=""><br class="">--<span class=""> </span><br class=""><div class=""><span name="x" class=""></span>Hal Finkel<br class="">Assistant Computational Scientist<br class="">Leadership Computing Facility<br class="">Argonne National Laboratory</div></div></div></blockquote></div></div>_______________________________________________<br class="">
LLVM Developers mailing list<br class="">
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class="">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">
</blockquote></div>
</div></blockquote></div><br class=""></body></html>