<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p><font face="Courier New, Courier, monospace"><br>
</font></p>
<p><font face="Courier New, Courier, monospace">Just as an aside, as
far as running a custom pass on testsuite, one solution that I
was using is to configure testsuite using <br>
</font></p>
<p><font face="Courier New, Courier, monospace">cmake CC=<path to
my_own_script> <br>
</font></p>
<p><font face="Courier New, Courier, monospace">where my_own_script
might contain something like: clang -Xclang -load -Xclang
mypass.so "$@"</font></p>
<p><font face="Courier New, Courier, monospace">Nevertheless, the
previous "make TEST=mypass" solution was much more handy in term
of the different ways you can build or report the results. <br>
</font></p>
<p><font face="Courier New, Courier, monospace">-</font></p>
<p><font face="Courier New, Courier, monospace">Sandeep <br>
</font>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</p>
<br>
<div class="moz-cite-prefix">On 11/24/2016 04:57 PM, Dasgupta,
Sandeep via llvm-dev wrote:<br>
</div>
<blockquote
cite="mid:3c0jbvbf49i2negd3ddkskbd.1480027742023@email.android.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div>Hello All</div>
<div>As John rightly pointed out, my question is about adding the
capability to build the testsuite differently and not about how
to add other testsuites. </div>
<div> For example before configure build were deprecated, we can
do something like make TEST=mypass, to build the testsuite
according to the makefile rules in TEST.mypass.Makefile and that
was handy if we want to test out own passes on the testsuite. To
achieve the same in the current cmake build system, I need to do
a hack solution as mentioned in my first post. Is there any
clean way to achieve the same.</div>
<div><br>
</div>
<div>Thanks </div>
<div>Sandeep</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div id="composer_signature">
<div style="font-size:85%; color:#575757">Sent via the Samsung
Galaxy Note® 3, an AT&T 4G LTE smartphone</div>
</div>
<br>
<br>
-------- Original message --------<br>
From: John Criswell <a class="moz-txt-link-rfc2396E" href="mailto:jtcriswel@gmail.com"><jtcriswel@gmail.com></a> <br>
Date: 11/24/2016 9:02 AM (GMT-06:00) <br>
To: Arnaud De Grandmaison <a class="moz-txt-link-rfc2396E" href="mailto:Arnaud.DeGrandmaison@arm.com"><Arnaud.DeGrandmaison@arm.com></a>,
"Dasgupta, Sandeep" <a class="moz-txt-link-rfc2396E" href="mailto:sdasgup3@illinois.edu"><sdasgup3@illinois.edu></a>
<br>
Cc: <a class="moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>, nd <a class="moz-txt-link-rfc2396E" href="mailto:nd@arm.com"><nd@arm.com></a> <br>
Subject: Re: [llvm-dev] Running "different tests" on cmake based
test-suite build
<br>
<br>
<div>
<div class="moz-cite-prefix">On 11/24/16 3:12 AM, Arnaud De
Grandmaison via llvm-dev wrote:<br>
</div>
<blockquote type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span>Hi Sandeep,</span></p>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span lang="EN-US">The CMake version of
the test-suite has been improved (see
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D21360&d=DQMDaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=at_zHdaJ5pMt_A20sKa_JLbvjeXndNos3PveVfgUF0o&m=elm0JHrSir46GeQfeVXpM9G9IL8qP9v_jqIz9nqEDrI&s=69KaBU9uj9j_L8_uw9gE8Vcq2CN4j7BH0vab0K97pI8&e=">
https://reviews.llvm.org/D21360</a>) so that it’s easy
to plug additional test suites. A starting point would
be to look at how the TEST_SUITE_SUBDIRS variable in the
top level CMakeLists.txt is used.</span></p>
</div>
</blockquote>
<br>
I think this bug report is about adding new programs to the test
suite. I don't think Sandeep is asking how to add more test
programs to the test suite; he is asking how to add new
Makefiles to the test suite to make it run new experiments that
it did not run previously.<br>
<br>
As context, several previous research projects used an interface
within the old autoconf test suite to extend it to compile
programs with different compiler options and run performance
experiments on them. For example, the SAFECode and Automatic
Pool Allocation projects added rules to compile and run programs
with additional transforms and to record the results in CSV
files which could be imported into other programs for creating
graphs for papers. All of this could be done without modifying
any files within the test-suite project; one simply created
additional Makefiles, set come variables on the make command
line, and fired off the experiments with "make."<br>
<br>
I believe Sandeep is asking whether similar functionality exists
within the new Cmake build system and, if so, how to use it. I
am curious, too. It would make running some experiments for my
research projects more convenient. I just assumed this feature
was deprecated as I got the impression that I was the only one
still using it.<br>
<br>
Regards,<br>
<br>
John Criswell<br>
<br>
<blockquote type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span>Kind regards,</span></p>
<p class="MsoNormal"><span>Arnaud</span></p>
<p class="MsoNormal"><span> </span></p>
<div>
<div>
<div>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> llvm-dev [<a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="mailto:llvm-dev-bounces@lists.llvm.org">mailto:llvm-dev-bounces@lists.llvm.org</a>]
<b>On Behalf Of </b>Sandeep Dasgupta via llvm-dev<br>
<b>Sent:</b> 24 November 2016 02:32<br>
<b>To:</b> <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:llvm-dev@lists.llvm.org">
llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> [llvm-dev] Running "different
tests" on cmake based test-suite build</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p><span>Hello Friends,</span></p>
<p><span>With configure based build deprecated on llvm,
how can I run different tests/pass on the suite (<a
moz-do-not-send="true"
href="https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_TestSuiteMakefileGuide.html-23running-2Ddifferent-2Dtests&d=DQMDaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=at_zHdaJ5pMt_A20sKa_JLbvjeXndNos3PveVfgUF0o&m=elm0JHrSir46GeQfeVXpM9G9IL8qP9v_jqIz9nqEDrI&s=Jykf6DJSLcnbNS1O4aEwAFiUFeyMcoQb8n0qOZfhnSc&e=">http://llvm.org/docs/TestSuiteMakefileGuide.html#running-different-tests</a>)?
</span></p>
<p><span>Right now I am having a hacky solution, which
includes running the "still present" configure file of
test-suite to generate the
MAKE.<MYPASS>.Makefile;
</span></p>
<blockquote>
<p><span>cd external-testsuite-build-loc</span></p>
<p><span><llvmsrc>/project/test-suite/configure
--enable-optimized -disable-assertions
</span></p>
</blockquote>
<p><span>but while running 'make TEST=PASS' I am getting
error for trying to include
$(LLVM_OBJ_ROOT)/Makefile.config which is obviously
not there on a cmake based llvm build. I have to edit
the generated makefile to bypass this.</span></p>
<p><span>Can you please point me to some clean solution or
let me know if am missing something?
</span></p>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">-- <br>
<b>With Thanks and Regards</b><br>
Sandeep Dasgupta<br>
Research Assistant CS @UIUC<br>
(<a moz-do-not-send="true"
href="http://web.engr.illinois.edu/%7Esdasgup3/">http://web.engr.illinois.edu/~sdasgup3/</a>)
</p>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
LLVM Developers mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DQMDaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=at_zHdaJ5pMt_A20sKa_JLbvjeXndNos3PveVfgUF0o&m=elm0JHrSir46GeQfeVXpM9G9IL8qP9v_jqIz9nqEDrI&s=-MxG49PzHWKO5EeBSgjRewqderemUUsFzJPt5vmmSOE&e=">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
<br>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
John Criswell
Assistant Professor
Department of Computer Science, University of Rochester
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://urldefense.proofpoint.com/v2/url?u=http-3A__www.cs.rochester.edu_u_criswell&d=DQMDaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=at_zHdaJ5pMt_A20sKa_JLbvjeXndNos3PveVfgUF0o&m=elm0JHrSir46GeQfeVXpM9G9IL8qP9v_jqIz9nqEDrI&s=b5hyuBgRB-MigEMyXeeDgzgAG8FwHHfVrL1oFF-d0Yw&e=">http://www.cs.rochester.edu/u/criswell</a></pre>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
LLVM Developers mailing list
<a class="moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>
<a class="moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<b>With Thanks and Regards</b><br>
Sandeep Dasgupta<br>
Research Assistant CS @UIUC<br>
(<a class="moz-txt-link-freetext" href="http://web.engr.illinois.edu/~sdasgup3/">http://web.engr.illinois.edu/~sdasgup3/</a>)
</div>
</body>
</html>