<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">On the original thread topic, in r238965 I committed a much better detection algorithm, which uses a min of diffs approach. And in r238968 I updated the daily report to pass more data to make use of this.  The change brings the false positive rate down to about ~15% on our internal reports.  For the last two weeks I have actually been able to detect real regression in the 1% range!<div class=""><br class=""></div><div class="">This approach only works when the previous samples set is large enough to have some meaningful previous information in it.  Call sites of the regression detection have to be changed to pass more data.  For the daily report I changed it from comparing last run on current and previous days, to comparing last run on first day to all runs on previous day.</div><div class=""><br class=""></div><div class="">If this works out well, lets consider changing Run comparisons and Field comparisons to work in a similar way. <br class=""><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jun 2, 2015, at 6:04 PM, Sean Silva <<a href="mailto:chisophugis@gmail.com" class="">chisophugis@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, Jun 2, 2015 at 2:24 PM, Philip Reames <span dir="ltr" class=""><<a href="mailto:listmail@philipreames.com" target="_blank" class="">listmail@philipreames.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000" class="">
    Personally, I would prefer this either live in it's own repository,
    or llvm/tools/.  None of my use cases will likely involve the
    test-suite.<br class="">
    <br class="">
    p.s. If this is going to end up an llvm tool, it will need to follow
    LLVM style.  <br class="">
    <br class="">
    p.p.s. We should probably start a new thread with the proposed
    addition since I imagine many folks are ignoring this one by now
    given how deep it's gotten.</div></blockquote><div class=""><br class=""></div><div class="">Maybe just putting it on github for now is easiest to at least make it generally available for review. If we later want to officially pull it in or integrate it with our build system we can do that.</div><div class=""><br class=""></div><div class="">-- Sean Silva</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000" class=""><span class="HOEnZb"><font color="#888888" class=""><br class="">
    <br class="">
    Philip</font></span><div class=""><div class="h5"><br class="">
    <br class="">
    <div class="">On 06/02/2015 12:04 PM, Chris Matthews
      wrote:<br class="">
    </div>
    <blockquote type="cite" class="">
      
      I like that idea!
      <div class=""><br class="">
      </div>
      <div class=""><br class="">
        <div class="">
          <div class="">
            <blockquote type="cite" class="">
              <div class="">On Jun 2, 2015, at 12:00 PM, Smith, Kevin B
                <<a href="mailto:kevin.b.smith@intel.com" target="_blank" class="">kevin.b.smith@intel.com</a>>
                wrote:</div>
              <br class="">
              <div class="">
                
                
                
                <div link="blue" vlink="purple" lang="EN-US" class="">
                  <div class=""><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">The code for cmpimage and getdep
                        consists of five source files, with the
                        following sizes<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">$ wc *<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">  5912  20353 191869 cmpimage.cpp<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">   290   1328  10668 elf.h<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">  1496   5006  41691 getdep.cpp<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">   233    959   7692 macho.h<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">   403   1831  18394 pecoff.h<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">  8334  29477 270314 total<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">to build each of them is just a simple
                        compilation for whatever C++ compiler you happen
                        to be using (clang, icc, cl, g++)<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">$(CXX) –o cmpimage –O2 cmpimage.cpp<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">$(CXX) –o getdep –O2 getdep.cpp<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">This seems like it would fit rather
                        easily into test-suite/tools, which already
                        exists and has a Makefile that the commands to
                        build<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">these could be integrated into.<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">This is my best guess/opinion based on
                        a cursory look over the test-suite directory
                        structure.<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">Kevin<u class=""></u><u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""><u class=""></u> <u class=""></u></span></p>
                    <div class="">
                      <div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in" class=""><p class="MsoNormal"><b class=""><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" class="">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" class=""> Chris Matthews [<a href="mailto:chris.matthews@apple.com" target="_blank" class="">mailto:chris.matthews@apple.com</a>]
                            <br class="">
                            <b class="">Sent:</b> Thursday, May 28, 2015
                            1:02 PM<br class="">
                            <b class="">To:</b> Smith, Kevin B<br class="">
                            <b class="">Cc:</b> Philip Reames; Sean
                            Silva; LLVM Developers Mailing List<br class="">
                            <b class="">Subject:</b> Re: [LLVMdev]
                            Proposal: change LNT’s regression detection
                            algorithm and how it is used to reduce false
                            positives<u class=""></u><u class=""></u></span></p>
                      </div>
                    </div><p class="MsoNormal"><u class=""></u> <u class=""></u></p><p class="MsoNormal">Where is the best place to keep
                      this?  <u class=""></u><u class=""></u></p>
                    <div class=""><p class="MsoNormal"><u class=""></u> <u class=""></u></p>
                    </div>
                    <div class=""><p class="MsoNormal">- As third party tool we all
                        use?<u class=""></u><u class=""></u></p>
                    </div>
                    <div class=""><p class="MsoNormal">- Contribute as new project?<u class=""></u><u class=""></u></p>
                    </div>
                    <div class=""><p class="MsoNormal">- Lives in test-suite/utils?<u class=""></u><u class=""></u></p>
                    </div>
                    <div class=""><p class="MsoNormal">- Lives in llvm/utils?<u class=""></u><u class=""></u></p>
                      <div class=""><p class="MsoNormal"><u class=""></u> <u class=""></u></p>
                        <div class="">
                          <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt" class="">
                            <div class=""><p class="MsoNormal">On May 28, 2015, at
                                11:22 AM, Smith, Kevin B <<a href="mailto:kevin.b.smith@intel.com" target="_blank" class="">kevin.b.smith@intel.com</a>>
                                wrote:<u class=""></u><u class=""></u></p>
                            </div><p class="MsoNormal"><u class=""></u> <u class=""></u></p>
                            <div class="">
                              <div class=""><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">OK, there is interest from
                                    at least a couple of people.  What
                                    should next steps be?</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""> </span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">Kevin</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""> </span><u class=""></u><u class=""></u></p>
                                <div class="">
                                  <div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in" class=""><p class="MsoNormal"><b class=""><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" class="">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" class=""> Chris Matthews [<a href="mailto:chris.matthews@apple.com" target="_blank" class="">mailto:chris.matthews@apple.com</a>]
                                        <br class="">
                                        <b class="">Sent:</b> Thursday,
                                        May 28, 2015 10:57 AM<br class="">
                                        <b class="">To:</b> Philip
                                        Reames<br class="">
                                        <b class="">Cc:</b> Smith, Kevin
                                        B; Sean Silva; LLVM Developers
                                        Mailing List<br class="">
                                        <b class="">Subject:</b> Re:
                                        [LLVMdev] Proposal: change LNT’s
                                        regression detection algorithm
                                        and how it is used to reduce
                                        false positives</span><u class=""></u><u class=""></u></p>
                                  </div>
                                </div><p class="MsoNormal"> <u class=""></u><u class=""></u></p><p class="MsoNormal">I agree. I think
                                  there are a lot of exciting uses for
                                  this tool.  A stage 3 build bot would
                                  be another one.<u class=""></u><u class=""></u></p>
                                <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                </div>
                                <div class="">
                                  <div class="">
                                    <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt" class="">
                                      <div class=""><p class="MsoNormal">On May 28,
                                          2015, at 10:14 AM, Philip
                                          Reames <<a href="mailto:listmail@philipreames.com" target="_blank" class="">listmail@philipreames.com</a>>
                                          wrote:<u class=""></u><u class=""></u></p>
                                      </div><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                      <div class="">
                                        <div class=""><p class="MsoNormal" style="margin-bottom:12.0pt">I'd
                                            love to see this tool
                                            contributed, even it isn't
                                            used for regression
                                            detection work.  I've got a
                                            couple of hacked up scripts
                                            which do similar things and
                                            having a robust tool
                                            available for this would be
                                            very useful.  <br class="">
                                            <br class="">
                                            Philip<u class=""></u><u class=""></u></p>
                                          <div class=""><p class="MsoNormal">On
                                              05/26/2015 09:53 AM,
                                              Smith, Kevin B wrote:<u class=""></u><u class=""></u></p>
                                          </div>
                                          <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt" class=""><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">Intel has a
                                                binary comparator tool
                                                that we have been using
                                                for several years for
                                                comparing output
                                                binaries</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">to see if the
                                                code within them is
                                                considered identical. 
                                                We use it to eliminate
                                                runs (and therefore some
                                                performance noise)</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">from our own
                                                performance tracking
                                                tools.</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""> </span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">We are willing
                                                to contribute the source
                                                code for this to the
                                                LLVM community if there
                                                is interest.</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""> </span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">There are two
                                                programs involved: 
                                                getdep, which displays
                                                the list of DLL/.so
                                                dependencies of the
                                                image in question, and
                                                cmpimage itself, which
                                                does the comparison
                                                ignoring the parts not
                                                contributed by the
                                                compiler.  The cmpimage
                                                program is also almost
                                                completely derived from
                                                the published object
                                                format descriptions.</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""> </span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">Let me know if
                                                there is interest in
                                                these pieces of tooling,
                                                and if so, what you
                                                think next steps should
                                                be.</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""> </span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class="">Kevin B. Smith</span><u class=""></u><u class=""></u></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1f497d" class=""> </span><u class=""></u><u class=""></u></p><p class="MsoNormal"><b class=""><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" class="">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" class="">
                                                <a href="mailto:llvmdev-bounces@cs.uiuc.edu" target="_blank" class="">llvmdev-bounces@cs.uiuc.edu</a>
                                                [<a href="mailto:llvmdev-bounces@cs.uiuc.edu" target="_blank" class="">mailto:llvmdev-bounces@cs.uiuc.edu</a>]
                                                <b class="">On Behalf Of
                                                </b>Sean Silva<br class="">
                                                <b class="">Sent:</b>
                                                Thursday, May 21, 2015
                                                2:14 PM<br class="">
                                                <b class="">To:</b>
                                                Chris Matthews<br class="">
                                                <b class="">Cc:</b> LLVM
                                                Developers Mailing List<br class="">
                                                <b class="">Subject:</b>
                                                Re: [LLVMdev] Proposal:
                                                change LNT’s regression
                                                detection algorithm and
                                                how it is used to reduce
                                                false positives</span><u class=""></u><u class=""></u></p><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                            <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                              <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                <div class=""><p class="MsoNormal">On
                                                    Thu, May 21, 2015 at
                                                    11:24 AM, Chris
                                                    Matthews <<a href="mailto:chris.matthews@apple.com" target="_blank" class="">chris.matthews@apple.com</a>>
                                                    wrote:<u class=""></u><u class=""></u></p><p class="MsoNormal">I
                                                    agree this is a
                                                    great idea.  I think
                                                    it needs to be
                                                    fleshed out a little
                                                    though.<br class="">
                                                    <br class="">
                                                    It would still be
                                                    wise to run the
                                                    regression detection
                                                    algorithm, because
                                                    the test suite
                                                    changes and the
                                                    machines change, and
                                                    the algorithm is not
                                                    perfect yet.  It
                                                    would be a valuable
                                                    source of
                                                    information though.<u class=""></u><u class=""></u></p>
                                                  <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal">How
                                                      would running it
                                                      as part of regular
                                                      testing change
                                                      anything?
                                                      Presumably the
                                                      only purpose it
                                                      would serve is
                                                      retrospectively
                                                      going back and
                                                      seeing
                                                      false-positives in
                                                      the aggregate. But
                                                      if we are already
                                                      doing offline
                                                      analysis, we can
                                                      run the regression
                                                      detection
                                                      algorithm (or any
                                                      prospective ones)
                                                      offline on the raw
                                                      data; it doesn't
                                                      take that long.<u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt" class=""><p class="MsoNormal"><br class="">
                                                      This is not a
                                                      small change to
                                                      how LNT works, so
                                                      I think some due
                                                      diligence is
                                                      necessary.  Is
                                                      clang *really*
                                                      that
                                                      deterministic,
                                                      especially over
                                                      successive revs? <u class=""></u><u class=""></u></p>
                                                  </blockquote>
                                                  <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal">Yes.
                                                      Actually, google's
                                                      build system
                                                      depends on this
                                                      for its caching
                                                      strategy to work
                                                      and so the google
                                                      guys are usually
                                                      on top of any
                                                      issues in this
                                                      respect (thanks
                                                      google guys!).<u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt" class=""><p class="MsoNormal">I
                                                      know it is
                                                      supposed to be. 
                                                      Does anyone have
                                                      any data to show
                                                      this is going to
                                                      be an effective
                                                      approach?  It
                                                      seems like there
                                                      are benchmarks in
                                                      the test-suite
                                                      which use __DATE__
                                                      and __TIME__ in
                                                      them. I assume
                                                      that will be a
                                                      problem?<u class=""></u><u class=""></u></p>
                                                  </blockquote>
                                                  <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal">__DATE__
                                                      and __TIME__
                                                      should be easy to
                                                      solve by modifying
                                                      the benchmark, or
                                                      teaching clang to
                                                      always return a
                                                      fixed value for
                                                      them (maybe we
                                                      already have this?
                                                      IIRC google's
                                                      build system does
                                                      something like
                                                      this; or maybe the
                                                      do it at the OS
                                                      level).<u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal">--
                                                      Sean Silva<u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <div class=""><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                                  </div>
                                                  <blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt" class="">
                                                    <div class="">
                                                      <div class=""><p class="MsoNormal"><br class="">
                                                          > On May
                                                          21, 2015, at
                                                          1:43 AM,
                                                          Renato Golin
                                                          <<a href="mailto:renato.golin@linaro.org" target="_blank" class="">renato.golin@linaro.org</a>>
                                                          wrote:<br class="">
                                                          ><br class="">
                                                          > On 20 May
                                                          2015 at 23:31,
                                                          Sean Silva
                                                          <<a href="mailto:chisophugis@gmail.com" target="_blank" class="">chisophugis@gmail.com</a>>
                                                          wrote:<br class="">
                                                          >> In
                                                          the last
                                                          10,000
                                                          revisions of
                                                          LLVM+Clang,
                                                          only 10
                                                          revisions
                                                          actually<br class="">
                                                          >>
                                                          caused the
                                                          binary of
                                                          MultiSource/Benchmarks/BitBench/five11
                                                          to change. So
                                                          if<br class="">
                                                          >> just
                                                          store a hash
                                                          of the binary
                                                          in the
                                                          database, we
                                                          should be able
                                                          to pool<br class="">
                                                          >> all
                                                          samples we
                                                          have collected
                                                          while the
                                                          binary is the
                                                          the same as it<br class="">
                                                          >>
                                                          currently is,
                                                          which will let
                                                          us use
                                                          significantly
                                                          more
                                                          datapoints for
                                                          the<br class="">
                                                          >>
                                                          reference.<br class="">
                                                          ><br class="">
                                                          > +1<br class="">
                                                          ><br class="">
                                                          ><br class="">
                                                          >> Also,
                                                          we can
                                                          trivially
                                                          eliminate
                                                          running the
                                                          regression
                                                          detection
                                                          algorithm<br class="">
                                                          >> if
                                                          the binary
                                                          hasn't
                                                          changed.<br class="">
                                                          ><br class="">
                                                          > +2!<br class="">
                                                          ><br class="">
                                                          > --renato<u class=""></u><u class=""></u></p>
                                                      </div>
                                                    </div>
                                                    <div class="">
                                                      <div class=""><p class="MsoNormal" style="margin-bottom:12.0pt">>
                                                          _______________________________________________<br class="">
                                                          > LLVM
                                                          Developers
                                                          mailing list<br class="">
                                                          > <a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank" class="">LLVMdev@cs.uiuc.edu</a>     
                                                             <a href="http://llvm.cs.uiuc.edu/" target="_blank" class="">http://llvm.cs.uiuc.edu</a><br class="">
                                                          > <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank" class="">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><u class=""></u><u class=""></u></p>
                                                      </div>
                                                    </div>
                                                  </blockquote>
                                                </div><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                              </div>
                                            </div><p class="MsoNormal"><br class="">
                                              <br class="">
                                              <br class="">
                                              <br class="">
                                              <u class=""></u><u class=""></u></p>
                                            <pre class="">_______________________________________________<u class=""></u><u class=""></u></pre>
                                            <pre class="">LLVM Developers mailing list<u class=""></u><u class=""></u></pre>
                                            <pre class=""><a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank" class="">LLVMdev@cs.uiuc.edu</a>         <a href="http://llvm.cs.uiuc.edu/" target="_blank" class="">http://llvm.cs.uiuc.edu</a><u class=""></u><u class=""></u></pre>
                                            <pre class=""><a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank" class="">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><u class=""></u><u class=""></u></pre>
                                          </blockquote><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                        </div>
                                      </div>
                                    </blockquote>
                                  </div><p class="MsoNormal"> <u class=""></u><u class=""></u></p>
                                </div>
                              </div>
                            </div>
                          </blockquote>
                        </div><p class="MsoNormal"><u class=""></u> <u class=""></u></p>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </blockquote>
          </div>
          <br class="">
        </div>
      </div>
    </blockquote>
    <br class="">
  </div></div></div>

</blockquote></div><br class=""></div></div>
</div></blockquote></div><br class=""></div></div></body></html>