<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi <span style="background-color: rgb(255, 255, 255);" class="">Siddharth</span><div class=""><span style="background-color: rgb(255, 255, 255);" class=""><br class=""></span></div><div class=""><span style="background-color: rgb(255, 255, 255);" class="">The easiest way to do that is to run from LNT. You can refer to the quick start guide Teresa linked in her previous reply. You just need to pass in the proper --cflags to build test suite with the configuration you want. There are more advanced options like using a cmake cache file, which works as well.</span></div><div class=""><span style="background-color: rgb(255, 255, 255);" class=""><br class=""></span></div><div class=""><span style="background-color: rgb(255, 255, 255);" class="">You can also search for LNT jobs on the bots to see how they are configured to run, for example:</span></div><div class=""><span style="background-color: rgb(255, 255, 255);" class=""><a href="http://lab.llvm.org:8011/builders/clang-cmake-armv7-lnt" class="">http://lab.llvm.org:8011/builders/clang-cmake-armv7-lnt</a></span></div><div class=""><span style="background-color: rgb(255, 255, 255);" class=""><a href="http://green.lab.llvm.org/green/job/lnt-test-suite-x86_64-O3-flto/2718/consoleFull" class="">http://green.lab.llvm.org/green/job/lnt-test-suite-x86_64-O3-flto/2718/consoleFull</a></span></div><div class=""><span style="background-color: rgb(255, 255, 255);" class=""><br class=""></span></div><div class=""><span style="background-color: rgb(255, 255, 255);" class=""><br class=""></span></div><div class=""><span style="background-color: rgb(255, 255, 255);" class="">Steven<br class=""></span><div><br class=""><blockquote type="cite" class=""><div class="">On May 11, 2018, at 8:39 AM, Teresa Johnson via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class="">Hopefully someone else on the dev list who has experience with LNT and the LLVM test-suite will chime in. I've never run it myself. Adding Mehdi since he ran it with LTO/ThinLTO.<br class=""><div class=""><br class=""></div><div class="">I found some documentation for the test-suite:</div><div class=""><a href="https://llvm.org/docs/TestingGuide.html#test-suite-overview" class="">https://llvm.org/docs/TestingGuide.html#test-suite-overview</a><br class=""></div><div class=""><a href="http://llvm.org/docs/lnt/quickstart.html" class="">http://llvm.org/docs/lnt/quickstart.html</a><br class=""></div><div class=""><a href="http://llvm.org/docs/lnt/tests.html#llvm-cmake-test-suite" class="">http://llvm.org/docs/lnt/tests.html#llvm-cmake-test-suite</a><br class=""></div><div class=""><br class=""></div><div class="">But I can't find any documentation describing how to pass custom options to lnt runtests, so hopefully Mehdi or someone else can help.</div><div class=""><br class=""></div><div class="">Teresa</div></div><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class=""><div class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;"><div dir="ltr" class="">On Fri, May 11, 2018 at 7:59 AM Siddharth Shankar Swain <<a href="mailto:h2015096@pilani.bits-pilani.ac.in" class="">h2015096@pilani.bits-pilani.ac.in</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class="">Yes running LLVM performance test suite with LTO and Thin LTO enabled.<div class="">Thanks,</div><div class="">Siddharth</div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Fri, May 11, 2018 at 7:32 PM, Teresa Johnson<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:tejohnson@google.com" target="_blank" class="">tejohnson@google.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class="">I'm not completely sure what you are asking. Are you looking for performance benchmarks to use for LTO and ThinLTO testing? Or are you asking how to build/run with LTO and ThinLTO? Are you asking how to run LLVM's performance test-suite with LTO and ThinLTO?<span class="m_437944333938055701HOEnZb"><font color="#888888" class=""><br class=""><div class=""><br class=""></div><div class="">Teresa</div></font></span></div><div class="m_437944333938055701HOEnZb"><div class="m_437944333938055701h5"><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Fri, May 11, 2018 at 5:21 AM Siddharth Shankar Swain <<a href="mailto:h2015096@pilani.bits-pilani.ac.in" target="_blank" class="">h2015096@pilani.bits-pilani.ac.in</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class="">Hi,<div class="">Thanks for the info, If i only want to run performance test on benchmarks for LTO and Thin LTO enabled target, Can u suggest ways to do it ? I want to do it at my end.</div><div class="">Thanks,</div><div class="">Siddharth</div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, May 2, 2018 at 7:03 PM, Teresa Johnson<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:tejohnson@google.com" target="_blank" class="">tejohnson@google.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class="">I've only measured performance on x86. There are some old results for SPEC cpu2006 in the blog post here:<div class=""> <a href="http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html" target="_blank" class="">http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html</a><br class=""></div><div class="">The benefit is benchmark dependent, e.g. small benchmarks often don't improve much as they don't require whole program optimizations.</div><div class=""><br class=""></div><div class="">We've improved the ThinLTO optimizations since then, but I don't have any current numbers for SPEC handy.</div><div class=""><br class=""></div><div class="">Note that the performance is higher if you combine with PGO as the two techniques are complementary. I would say 5-10% improvement is pretty common (when measured on top of PGO), for codes that can benefit from whole program / cross-module optimizations.</div><div class=""><br class=""></div><div class="">Teresa</div></div><div class="gmail_extra"><br class=""><div class="gmail_quote"><div class=""><div class="m_437944333938055701m_-2332319272784075004m_8492934858772635264h5">On Tue, May 1, 2018 at 12:12 PM, Siddharth Shankar Swain via llvm-dev<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""></div></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class="m_437944333938055701m_-2332319272784075004m_8492934858772635264h5"><div dir="ltr" class="">Hi all,<div class=""><div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; background-color: rgb(255, 255, 255);" class="">We have enabled Thin LTO and LTO for a specific target architecture. What can be the possible scopes of improvement depending on the target after we enable the basic LTO and thin LTO.? </div><div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; background-color: rgb(255, 255, 255);" class="">Thanks,</div><div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; background-color: rgb(255, 255, 255);" class="">Siddharth</div><br class=""></div></div><br class=""></div></div>_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class=""><br class=""></blockquote></div><span class="m_437944333938055701m_-2332319272784075004m_8492934858772635264HOEnZb"><font color="#888888" class=""><br class=""><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div class="m_437944333938055701m_-2332319272784075004m_8492934858772635264m_2842654232110643302gmail_signature" data-smartmail="gmail_signature"><span style="font-family: Times; font-size: inherit;" class=""><table cellspacing="0" cellpadding="0" class=""><tbody class=""><tr style="color: rgb(85, 85, 85); font-family: sans-serif; font-size: small;" class=""><td nowrap="" style="border-top-style: solid; border-top-color: rgb(213, 15, 37); border-top-width: 2px;" class="">Teresa Johnson |</td><td nowrap="" style="border-top-style: solid; border-top-color: rgb(51, 105, 232); border-top-width: 2px;" class=""> Software Engineer |</td><td nowrap="" style="border-top-style: solid; border-top-color: rgb(0, 153, 57); border-top-width: 2px;" class=""> <a href="mailto:tejohnson@google.com" target="_blank" class="">tejohnson@google.com</a> |</td><td nowrap="" style="border-top-style: solid; border-top-color: rgb(238, 178, 17); border-top-width: 2px;" class=""> 408-460-2413</td></tr></tbody></table></span></div></font></span></div></blockquote></div><br class=""></div></blockquote></div><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div dir="ltr" class="m_437944333938055701m_-2332319272784075004gmail_signature" data-smartmail="gmail_signature"><span style="font-family: Times; font-size: inherit;" class=""><table cellspacing="0" cellpadding="0" class=""><tbody class=""><tr style="color: rgb(85, 85, 85); font-family: sans-serif; font-size: small;" class=""><td style="border-top-style: solid; border-top-color: rgb(213, 15, 37); border-top-width: 2px;" class="">Teresa Johnson |</td><td style="border-top-style: solid; border-top-color: rgb(51, 105, 232); border-top-width: 2px;" class=""> Software Engineer |</td><td style="border-top-style: solid; border-top-color: rgb(0, 153, 57); border-top-width: 2px;" class=""> <a href="mailto:tejohnson@google.com" target="_blank" class="">tejohnson@google.com</a> |</td><td style="border-top-style: solid; border-top-color: rgb(238, 178, 17); border-top-width: 2px;" class=""> 408-460-2413</td></tr></tbody></table></span></div></div></div></blockquote></div><br class=""></div></blockquote></div><br clear="all" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class=""><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class=""><br class=""></div><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none; float: none; display: inline !important;" class="">--<span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class=""><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;"><span style="font-family: Times; font-size: inherit;" class=""><table cellspacing="0" cellpadding="0" class=""><tbody class=""><tr style="color: rgb(85, 85, 85); font-family: sans-serif; font-size: small;" class=""><td style="border-top-style: solid; border-top-color: rgb(213, 15, 37); border-top-width: 2px;" class="">Teresa Johnson |</td><td style="border-top-style: solid; border-top-color: rgb(51, 105, 232); border-top-width: 2px;" class=""> Software Engineer |</td><td style="border-top-style: solid; border-top-color: rgb(0, 153, 57); border-top-width: 2px;" class=""> <a href="mailto:tejohnson@google.com" target="_blank" class="">tejohnson@google.com</a> |</td><td style="border-top-style: solid; border-top-color: rgb(238, 178, 17); border-top-width: 2px;" class=""> <a href="tel:408-460-2413" class="">408-460-2413</a></td></tr></tbody></table></span></div><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none; float: none; display: inline !important;" class="">LLVM Developers mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class="">llvm-dev@lists.llvm.org</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration: none;" class=""></div></blockquote></div><br class=""></div></body></html>