[llvm-dev] Running "different tests" on cmake based test-suite build

Thu Nov 24 18:58:29 PST 2016

Just as an aside, as far as running a custom pass on testsuite, one 
solution that I was using is to configure testsuite using

cmake CC=<path to my_own_script>

where my_own_script might contain something like: clang -Xclang -load 
-Xclang mypass.so "$@"

Nevertheless,  the previous "make TEST=mypass" solution was much more 
handy in term of the different ways you can build or report the results.

-

Sandeep

On 11/24/2016 04:57 PM, Dasgupta, Sandeep via llvm-dev wrote:
> Hello All
> As John rightly pointed out, my question is about adding the 
> capability to build the testsuite differently and not about how to add 
> other testsuites.
>  For example before configure build were deprecated, we can do 
> something like make TEST=mypass, to build the testsuite according to 
> the makefile rules in TEST.mypass.Makefile and that was handy if we 
> want to test out own passes on the testsuite. To achieve the same in 
> the current cmake build system, I need to do a hack solution as 
> mentioned in my first post. Is there any clean way to achieve the same.
>
> Thanks
> Sandeep
>
>
>
> Sent via the Samsung Galaxy Note® 3, an AT&T 4G LTE smartphone
>
>
> -------- Original message --------
> From: John Criswell <jtcriswel at gmail.com>
> Date: 11/24/2016 9:02 AM (GMT-06:00)
> To: Arnaud De Grandmaison <Arnaud.DeGrandmaison at arm.com>, "Dasgupta, 
> Sandeep" <sdasgup3 at illinois.edu>
> Cc: llvm-dev at lists.llvm.org, nd <nd at arm.com>
> Subject: Re: [llvm-dev] Running "different tests" on cmake based 
> test-suite build
>
> On 11/24/16 3:12 AM, Arnaud De Grandmaison via llvm-dev wrote:
>>
>> Hi Sandeep,
>>
>> The CMake version of the test-suite has been improved (see 
>> https://reviews.llvm.org/D21360) so that it’s easy to plug additional 
>> test suites. A starting point would be to look at how the 
>> TEST_SUITE_SUBDIRS variable in the top level CMakeLists.txt is used.
>>
>
> I think this bug report is about adding new programs to the test 
> suite.  I don't think Sandeep is asking how to add more test programs 
> to the test suite; he is asking how to add new Makefiles to the test 
> suite to make it run new experiments that it did not run previously.
>
> As context, several previous research projects used an interface 
> within the old autoconf test suite to extend it to compile programs 
> with different compiler options and run performance experiments on 
> them.  For example, the SAFECode and Automatic Pool Allocation 
> projects added rules to compile and run programs with additional 
> transforms and to record the results in CSV files which could be 
> imported into other programs for creating graphs for papers.  All of 
> this could be done without modifying any files within the test-suite 
> project; one simply created additional Makefiles, set come variables 
> on the make command line, and fired off the experiments with "make."
>
> I believe Sandeep is asking whether similar functionality exists 
> within the new Cmake build system and, if so, how to use it.  I am 
> curious, too.  It would make running some experiments for my research 
> projects more convenient.  I just assumed this feature was deprecated 
> as I got the impression that I was the only one still using it.
>
> Regards,
>
> John Criswell
>
>> Kind regards,
>>
>> Arnaud
>>
>> *From:*llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf 
>> Of *Sandeep Dasgupta via llvm-dev
>> *Sent:* 24 November 2016 02:32
>> *To:* llvm-dev at lists.llvm.org
>> *Subject:* [llvm-dev] Running "different tests" on cmake based 
>> test-suite build
>>
>> Hello Friends,
>>
>> With configure based build deprecated on llvm, how can I run 
>> different tests/pass on the suite 
>> (http://llvm.org/docs/TestSuiteMakefileGuide.html#running-different-tests 
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_TestSuiteMakefileGuide.html-23running-2Ddifferent-2Dtests&d=DQMDaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=at_zHdaJ5pMt_A20sKa_JLbvjeXndNos3PveVfgUF0o&m=elm0JHrSir46GeQfeVXpM9G9IL8qP9v_jqIz9nqEDrI&s=Jykf6DJSLcnbNS1O4aEwAFiUFeyMcoQb8n0qOZfhnSc&e=>)? 
>>
>>
>> Right now I am having a hacky solution, which includes running the 
>> "still present" configure file of test-suite to generate the 
>> MAKE.<MYPASS>.Makefile;
>>
>>     cd external-testsuite-build-loc
>>
>>     <llvmsrc>/project/test-suite/configure --enable-optimized
>>     -disable-assertions
>>
>> but while running 'make TEST=PASS' I am getting error for trying to 
>> include $(LLVM_OBJ_ROOT)/Makefile.config which is obviously not there 
>> on a cmake based llvm build. I have to edit the generated makefile to 
>> bypass this.
>>
>> Can you please point me to some clean solution or let me know if am 
>> missing something?
>>
>> -- 
>> *With Thanks and Regards*
>> Sandeep Dasgupta
>> Research Assistant CS @UIUC
>> (http://web.engr.illinois.edu/~sdasgup3/ 
>> <http://web.engr.illinois.edu/%7Esdasgup3/>)
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> -- 
> John Criswell
> Assistant Professor
> Department of Computer Science, University of Rochester
> http://www.cs.rochester.edu/u/criswell
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 
*With Thanks and Regards*
Sandeep Dasgupta
Research Assistant CS @UIUC
(http://web.engr.illinois.edu/~sdasgup3/)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20161124/d6eb680c/attachment-0001.html>