[llvm-commits] Fwd: Help with buildbot commit (was [PATCH] Change buildbot to patch lldb local changes)
Galina Kistanova
gkistanova at gmail.com
Fri Jan 27 11:18:41 PST 2012
Hi Mark,
Thanks for working on this!
Please CC the llvm-commits at cs.uiuc.edu with your patches.
The patch looks Ok to me.
The only thing is the configure without any explicit parameter. Would it
always choose the right things by default?
I'll commit this patch on Monday if no one will comment/object till then.
Thanks
Galina
---------- Forwarded message ----------
From: Mark Peek <mark at peek.org>
Date: Wed, Jan 25, 2012 at 9:22 PM
Subject: Re: Help with buildbot commit (was [PATCH] Change buildbot to
patch lldb local changes)
To: Galina Kistanova <gkistanova at gmail.com>
Here's a version that appears to work both in and out of tree. I ran it
twice out of tree and then twice in tree to verify.
Thanks,
Mark
On 1/25/12 5:34 PM, Galina Kistanova wrote:
> That should be about the same. Just create a separate build/obj directory
> and
> run configure from there as ../<src>/configure <all params you need>.
> If both in and out will not work for any reason, I think, out of tree is
> preferable.
> Good luck.
> Thanks
> Galina
>
>
> On Wed, Jan 25, 2012 at 1:51 PM, Mark Peek <mark at peek.org
> <mailto:mark at peek.org>> wrote:
>
> The really short answer is because I didn't know any better about that
> being the recommended way. Looking at the script it appears the in/out
> is
> dictated by how the factory is being called and most of the logic is
> there. But there might be a bug in it where the configure really needs
> to
> take into account the out of tree build. Let me take a look, see if I
> can
> get both in/out of tree working, and then get you a new patch.
>
> Mark
>
>
> On 1/25/12 12:56 PM, Galina Kistanova wrote:
>
> Want to ask why do you build in tree? The recommended way is to
> build out of
> tree.
>
> Thanks
>
> Galina
> On Tue, Jan 24, 2012 at 10:28 PM, Mark Peek <mark at peek.org
> <mailto:mark at peek.org>
> <mailto:mark at peek.org <mailto:mark at peek.org>>> wrote:
>
> Galina,
> Attached is an updated patch which will remove the source,
> checkout, patch
> and build lldb. Passed twice in a row in my buildbot setup to
> make
> sure it
> worked correctly with a verification that the source directory
> got
> removed
> in between runs.
>
> Thanks,
> Mark
>
>
> On 1/24/12 9:28 AM, Mark Peek wrote:
>
> Hi Galina,
> Thank you for the response. I hadn't realized there was an
> issue with
> svn revert and was trying to reduce the load on the svn
> server. Let me
> try redoing the patch to take that into account. Given my
> work
> schedule I'll see if I can get to it later this week or the
> weekend at
> the very latest.
>
> Thanks,
> Mark
>
> On 1/23/12 1:35 PM, Galina Kistanova wrote:
>
> Hi Mark,
> Maybe it makes sense to remove the source and check out
> the clean
> specific
> revision instead of reverting lldb patches?
> Unfortunatelly, svn
> revert doesn’t
> always do the job. The rest looks good enough.
> Thanks
> Galina
>
> On Sun, Jan 22, 2012 at 8:11 PM, Mark Peek <
> mark at peek.org
> <mailto:mark at peek.org>
> <mailto:mark at peek.org <mailto:mark at peek.org>>
> <mailto:mark at peek.org <mailto:mark at peek.org> <mailto:mark at peek.org
> <mailto:mark at peek.org>>>> wrote:
>
> Hi Galina,
> I posted the below change to lldb-commit that should
> help lldb
> start
> passing with the buildbot. But I haven't had any
> luck
> getting
> someone to
> pick it up and commit it. I did follow up with both
> the last
> person to
> commit to it and to the lldb-dev list.
>
> Would you be able to look at it and commit it? Or be
> able to
> suggest
> someone that would be able to?
>
> Thanks,
> Mark
>
> -------- Original Message --------
> Subject: [Lldb-commits] [PATCH] Change buildbot to
> patch lldb
> local changes
> Date: Sun, 08 Jan 2012 10:46:48 -0800
> From: Mark Peek <mark at peek.org <mailto:
> mark at peek.org>
> <mailto:mark at peek.org <mailto:mark at peek.org>>
> <mailto:mark at peek.org <mailto:mark at peek.org> <mailto:mark at peek.org
> <mailto:mark at peek.org>>>>
> To: lldb-commits at cs.uiuc.edu
> <mailto:lldb-commits at cs.uiuc.**edu <lldb-commits at cs.uiuc.edu>>
> <mailto:lldb-commits at cs.uiuc._**_edu
> <mailto:lldb-commits at cs.uiuc.**edu <lldb-commits at cs.uiuc.edu>>>
> <mailto:lldb-commits at cs.uiuc. <mailto:lldb-commits at cs.uiuc.>**____edu
>
>
> <mailto:lldb-commits at cs.uiuc._**_edu <mailto:lldb-commits at cs.uiuc.*
> *edu <lldb-commits at cs.uiuc.edu>>>>
>
> The buildbot script for lldb does not apply local
> lldb
> patches
> from the
> lldb/scripts directory which is currently causing
> compilation
> errors and
> failing the build. This patch will revert and
> reapply
> the lldb
> patches prior
> to building to provide a more consistent build.
>
> Note: this does stub out the test phase since that
> is
> hanging
> on my local
> buildbot system. But it is probably better to get
> the
> compile
> running
> again on the buildbot to find build regressions and
> then get
> the tests
> running later.
>
> Mark
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20120127/081c52e0/attachment.html>
-------------- next part --------------
Index: LLDBBuilder.py
===================================================================
--- LLDBBuilder.py (revision 148928)
+++ LLDBBuilder.py (working copy)
@@ -3,91 +3,112 @@
import buildbot
import buildbot.process.factory
from buildbot.steps.source import SVN
-from buildbot.steps.shell import SetProperty, ShellCommand, WarningCountingShellCommand
+from buildbot.steps.shell import Configure, SetProperty
+from buildbot.steps.shell import ShellCommand, WarningCountingShellCommand
from buildbot.process.properties import WithProperties
-import ClangBuilder
-
-def isNewLLVMRevision(build_status):
- if build_status.getNumber() == 0:
- return true
-
- current_llvmrev = build_status.getProperty('llvmrev')
- try:
- prev_build_no = build_status.getNumber()-1
- prev_build_status = build_status.getBuilder().getBuild(prev_build_no)
- prev_llvmrev = prev_build_status.getProperty('llvmrev')
- return prev_llvmrev != current_llvmrev
- except IndexError:
- return true
-
-def getLLDBBuildFactory(triple, outOfDir=False, useTwoStage=False,
+def getLLDBBuildFactory(triple, outOfDir=False, useTwoStage=False, jobs=1,
always_install=False, extra_configure_args=[],
env={}, *args, **kwargs):
- # FIXME: this code is copied from getClangBuildFactory
+
inDir = not outOfDir and not useTwoStage
if inDir:
llvm_srcdir = "llvm"
- llvm_1_objdir = "llvm"
- if always_install:
- llvm_1_installdir = "llvm.install"
- else:
- llvm_1_installdir = None
+ llvm_objdir = "llvm"
else:
llvm_srcdir = "llvm.src"
- llvm_1_objdir = "llvm.obj"
- llvm_1_installdir = "llvm.install.1"
- llvm_2_objdir = "llvm.obj.2"
- llvm_2_installdir = "llvm.install"
+ llvm_objdir = "llvm.obj"
f = buildbot.process.factory.BuildFactory()
+ # Determine the build directory.
+ f.addStep(buildbot.steps.shell.SetProperty(name="get_builddir",
+ command=["pwd"],
+ property="builddir",
+ description="set build dir",
+ workdir="."))
+
+ # We really want to revert the patched llvm/clang files but svn sometimes
+ # doesn't do the right thing. We're left with removing and rebuilding.
+ f.addStep(ShellCommand(name='rm-%s' % llvm_srcdir,
+ command=['rm', '-rf', llvm_srcdir],
+ haltOnFailure = True,
+ workdir='.', env=env))
+ # Find out what version of llvm and clang are needed to build this version
+ # of lldb. Right now we will assume they use the same version.
+ # XXX - could this be done directly on the master instead of the slave?
+ f.addStep(SetProperty(command='svn cat http://llvm.org/svn/llvm-project/lldb/trunk/scripts/build-llvm.pl | grep ^our.*llvm_revision | cut -d \\" -f 2',
+ property='llvmrev'))
+
+ # The SVN build step provides no mechanism to check out a specific revision
+ # based on a property, so just run the commands directly here.
+
+ svn_co = ['svn', 'checkout', '--force']
+ svn_co += ['--revision', WithProperties('%(llvmrev)s')]
+
+ # build llvm svn checkout command
+ svn_co_llvm = svn_co + \
+ [WithProperties('http://llvm.org/svn/llvm-project/llvm/trunk@%(llvmrev)s'),
+ llvm_srcdir]
+ # build clang svn checkout command
+ svn_co_clang = svn_co + \
+ [WithProperties('http://llvm.org/svn/llvm-project/cfe/trunk@%(llvmrev)s'),
+ '%s/tools/clang' % llvm_srcdir]
+
+ f.addStep(ShellCommand(name='svn-llvm',
+ command=svn_co_llvm,
+ haltOnFailure=True,
+ workdir='.'))
+ f.addStep(ShellCommand(name='svn-clang',
+ command=svn_co_clang,
+ haltOnFailure=True,
+ workdir='.'))
+
f.addStep(SVN(name='svn-lldb',
mode='update',
baseURL='http://llvm.org/svn/llvm-project/lldb/',
defaultBranch='trunk',
+ always_purge=True,
workdir='%s/tools/lldb' % llvm_srcdir))
- f.addStep(SetProperty(command='grep ^our.*llvm_revision scripts/build-llvm.pl | cut -d \\" -f 2',
- property='llvmrev',
- workdir='%s/tools/lldb' % llvm_srcdir))
- same_llvmrev = lambda step: not isNewLLVMRevision(step.build.getStatus())
- new_llvmrev = lambda step: isNewLLVMRevision(step.build.getStatus())
+ # Patch llvm with lldb changes
+ f.addStep(ShellCommand(name='patch.llvm',
+ command='for i in tools/lldb/scripts/llvm*.diff; do echo "Patching with file $i"; patch -p0 -i $i; done',
+ workdir=llvm_srcdir))
- # Clean LLVM only if its revision number changed since the last build.
- # Otherwise, only clean LLDB.
- clean_lldb = \
- WarningCountingShellCommand(name="clean-lldb",
- command=['make', "clean"],
- haltOnFailure=True,
- description="cleaning lldb",
- descriptionDone="clean lldb",
- workdir='%s/tools/lldb' % llvm_1_objdir,
- doStepIf=same_llvmrev)
+ # Patch clang with lldb changes
+ f.addStep(ShellCommand(name='patch.clang',
+ command='for i in ../lldb/scripts/clang*.diff; do echo "Patching with file $i"; patch -p0 -i $i; done',
+ workdir='%s/tools/clang' % llvm_srcdir))
- # We use force_checkout to ensure the initial checkout is not aborted due to
- # the presence of the tools/lldb directory
- clangf = ClangBuilder.getClangBuildFactory(triple, test=False,
- outOfDir=outOfDir,
- useTwoStage=useTwoStage,
- always_install=always_install,
- extra_configure_args=
- extra_configure_args+
- ['--enable-targets=host'],
- env=env,
- trunk_revision='%(llvmrev)s',
- force_checkout=True,
- clean=new_llvmrev,
- extra_clean_step=clean_lldb,
- *args, **kwargs)
- f.steps += clangf.steps
+ # Run configure
+ config_args = [WithProperties("%%(builddir)s/%s/configure" % llvm_srcdir),]
+ f.addStep(Configure(name='configure',
+ command=config_args,
+ workdir=llvm_objdir))
+ f.addStep(WarningCountingShellCommand(name="compile",
+ command=['nice', '-n', '10',
+ 'make', WithProperties("-j%s" % jobs)
+ ],
+ haltOnFailure=True,
+ workdir=llvm_objdir))
+
+ # The tests are hanging on Linux at the moment due to some "expect"
+ # functionality not happening correctly. For now we will stub out the tests
+ # so we can at least get builds running and reinstate the tests later.
+
+ # Fixup file needed for tests
+ # f.addStep(ShellCommand(name"copy-gnu_libstdcpp.py",
+ # command="cp tools/lldb/examples/synthetic/gnu_libstdcpp.py Debug+Asserts/bin",
+ # workdir=llvm_srcdir))
+
# Test.
- f.addStep(ShellCommand(name="test",
- command=['nice', '-n', '10',
- 'make'],
- haltOnFailure=True, description="test lldb",
- env=env,
- workdir='%s/tools/lldb/test' % llvm_1_objdir))
+ #f.addStep(ShellCommand(name="test",
+ # command=['nice', '-n', '10',
+ # 'make'],
+ # haltOnFailure=True, description="test lldb",
+ # env=env,
+ # workdir='%s/tools/lldb/test' % llvm_objdir))
return f
More information about the llvm-commits
mailing list