<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">There are a lot more things in there than the metadata detection. Off the top of my head:</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> - xml test reports for the CI systems, and CSV reports</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> - profile data collection</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> - test-suite json is different than LNTs</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> - compile/exec multisample</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> - the diagnose stuff</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> - running with PGO</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> - result filtering (exclude stats)</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> </div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">My goal is no client side LNT at all. Workflow: user checks out test-suite, runs, and curls the resulting json to the server. We use this workflow on all of our other LNT based data collection, and it works really well. Running test-suite manually, then some other tools after would be a regression in the complexity.</div> <div><br></div><div>For metadata: could we write a tool, that the test-suite runs ad the end, that makes a new json for submission? We could add the metadata and apply the transformations to make the json grockable by LNT, and produce the xml and csv. </div> <div id="bloop_sign_1485372112364166912" class="bloop_sign"></div> <br><p class="airmail_on">On January 25, 2017 at 11:14:06 AM, Matthias Braun (<a href="mailto:matze@braunis.de">matze@braunis.de</a>) wrote:</p> <blockquote type="cite" class="clean_bq"><span><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div></div><div>
<title></title>
<div class="">I did a half-hearted attempt here: <a href="https://reviews.llvm.org/D19949" class="">https://reviews.llvm.org/D19949</a></div>
<div class=""><br class=""></div>
<div class="">The main blocker here IMO is that we want some meta
information about a build in LNT (compiler version used, hostname
of the machine, ...) which the testsuite currently doesn't produce
by itself. I see 3 solutions for that:</div>
<div class="">- We could change the test-suite cmake file to write
this information to a separate file</div>
<div class="">- We could modify lit to produce the metainformation
in the output (that functionality feels out of place in lit IMO),
or possibly pass the information along from a file produced by
cmake</div>
<div class="">- We could continue using a client side lnt tool that
collects the information as part of the submission process.</div>
<div class=""><br class=""></div>
<div class="">- Matthias</div>
<div class=""><br class=""></div>
<div>
<blockquote type="cite" class="">
<div class="">On Jan 25, 2017, at 10:59 AM, Chris Matthews via
llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org" class="">llvm-commits@lists.llvm.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div id="bloop_customfont" style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">I agree that dropping all the client side tools would be
(REALLY) nice. The proxied test-suite approach adds
conceptual and implementation complexity.</div>
<br style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<div id="bloop_sign_1485370178489704192" class="bloop_sign" style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
</div>
<span style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">lnt runtest test-suite is still doing a lot, it is not by
any means a thin wrapper. I’d love to see a concrete proposal
how we can migrate that into the test suite.</span>
<div style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div>
<div style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Also at the same time, I think the test-suite would start
to need its own testing.</div>
<div style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<p class="airmail_on">On January 25, 2017 at 1:40:03 AM, Kristof
Beyls via Phabricator (<a href="mailto:reviews@reviews.llvm.org" class="">reviews@reviews.llvm.org</a>) wrote:</p>
<blockquote type="cite" class="clean_bq">
<div class="">
<div class=""></div>
<div class=""><span class="">kristof.beyls marked an inline comment
as done.<span class="Apple-converted-space"> </span><br class="">
kristof.beyls added a comment.<span class="Apple-converted-space"> </span><br class="">
<br class="">
<br class="">
<br class="">
>>> As for the rest of the commit: Using a toolchain file
looks like just passing a -D flag which is already supported. As
for picking up the compiler, couldn't you simply extract that from
CMakeCache.txt? And if that doesn't work, shouldn't we rather
upstream the changes to CMakeLists.txt that print the compiler
instead of having LNT mess around with them?<span class="Apple-converted-space"> </span><br class="">
>><span class="Apple-converted-space"> </span><span class="Apple-converted-space"> </span><br class="">
>> I'm trying to eliminate as many not-needed command line
options as possible, e.g. --llvm-arch, --cross-compiling, ... .
There are already more than enough command line options to lnt
runtest, reducing the amount of them probably would make it simpler
to use.<span class="Apple-converted-space"> </span><br class="">
>> Indeed, just using a -D flag to enable cross-compiling
instead of having a separate lnt runtest command line option for it
would be preferable (and then documenting how to use it to do
cross-compiling).<span class="Apple-converted-space"> </span><br class="">
>> But then, doesn't the same argument apply to dropping lnt
runtest -cflags or lnt runtest -cppflags: they also both could be
set using -D?<span class="Apple-converted-space"> </span><br class="">
>> Anyway, let's not go there as part of this patch; I'll
first look into whether I can eliminate the need for <a href="http://opts.cc" class="">opts.cc</a> and opts.cxx also for the
non-cross-compiling case. In that case, I indeed no longer see the
need for a separate option so that lnt is aware we're
cross-compiling, and instead we can just use -D.<span class="Apple-converted-space"> </span><br class="">
><span class="Apple-converted-space"> </span><span class="Apple-converted-space"> </span><br class="">
> My argument would be to drop the whole "lnt runtest" mode and
let users do the cmake/ninja/lit sequence themselfes and teach "lnt
submit" to just submit the results. That way you are never limited
by missing or confusing options in lnt runtest, have one layer less
to debug and we have one thing less to maintain and keep up to
date. Though we may need to write more documentation about the
cmake/lit test-suite.<span class="Apple-converted-space"> </span><br class="">
<br class="">
Interesting idea, I like it! When I explain others
(non-LLVM-developers) about what lnt is, the first thing I explain
is that it is both a tool to run the LLVM test-suite (which mostly
isn't interesting to the non-LLVM-developers) and a
server/database/analysis engine to keep track of code generation
quality of a code generator (which is interesting to non-LLVM code
generator developers). At least from this point of view, it'd be
nice if lnt was "one thing" rather than "two things".<span class="Apple-converted-space"> </span><br class="">
Maybe 'lnt submit' could also be dropped, and users could upload
the produced json after a lit run using e.g. curl? On one hand,
submitting jsons to the server could be classified as being part of
the "server/database/analysis engine" functionality. OTOH, it might
remove the need for lnt on the client/test machine side
completely?<span class="Apple-converted-space"> </span><br class="">
<br class="">
<br class="">
<a href="https://reviews.llvm.org/D29030" class="">https://reviews.llvm.org/D29030</a><span class="Apple-converted-space"> </span><br class="">
<br class="">
<br class="">
<br class=""></span></div>
</div>
</blockquote>
</div>
<span style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">llvm-commits mailing list</span><br style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="mailto:llvm-commits@lists.llvm.org" style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">llvm-commits@lists.llvm.org</a><br style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></div>
</blockquote>
</div>
<br class="">
</div></div></span></blockquote></body></html>