Add ZUUL_COMMIT.
To facilitate using zuul with just the jenkins git plugin, add the ZUUL_COMMIT parameter. Also, always include ZUUL_REF, whether the job is pre- or post-commit. If it's pre, it will be a refs/zuul ref, if it's post, it will be refs/tags or a branch name. Change-Id: I88c38a28dcd552b2540095d36caacd10acf167b8 Reviewed-on: https://review.openstack.org/13934 Reviewed-by: Clark Boylan <clark.boylan@gmail.com> Approved: James E. Blair <corvus@inaugust.com> Tested-by: Jenkins
This commit is contained in:
parent
4aa1ad6a8e
commit
81515adbd6
|
@ -63,44 +63,71 @@ Zuul will pass some parameters to Jenkins for every job it launches.
|
|||
Check **This build is parameterized**, and add the following fields
|
||||
with the type **String Parameter**:
|
||||
|
||||
**UUID**
|
||||
**ZUUL_UUID**
|
||||
Zuul provided key to link builds with Gerrit events
|
||||
**GERRIT_PROJECT**
|
||||
Zuul provided project name
|
||||
**GERRIT_BRANCH**
|
||||
Zuul provided branch name
|
||||
**GERRIT_CHANGES**
|
||||
Zuul provided list of dependent changes to merge
|
||||
**ZUUL_REF**
|
||||
Zuul provided ref that includes commit(s) to build
|
||||
**ZUUL_COMMIT**
|
||||
The commit SHA1 at the head of ZUUL_REF
|
||||
|
||||
You may find it useful to use the ``GERRIT_*`` variables in your job.
|
||||
In particular, ``GERRIT_CHANGES`` indicates the change or changes that
|
||||
should be tested. If Zuul has decided that more than one change
|
||||
should be merged and tested together, they will all be listed in
|
||||
``GERRIT_CHANGES``. The format for the description of one change is::
|
||||
Those are the only required parameters. The UUID is needed for Zuul
|
||||
to keep track of the build, and the REF and COMMIT parameters are for
|
||||
use in preparing the git repo for the build. The following parameters
|
||||
will be sent for all builds, but are not required so you do not need
|
||||
to configure Jenkins to accept them if you do not plan on using them:
|
||||
|
||||
project:branch:refspec
|
||||
**ZUUL_PROJECT**
|
||||
The project that triggered this build
|
||||
**ZUUL_PIPELINE**
|
||||
The Zuul pipeline that is building this job
|
||||
|
||||
And multiple changes are separated by a carat ("^"). E.g.::
|
||||
The following parameters are optional and will only be provided for
|
||||
builds associated with changes (i.e., in response to patchset-created
|
||||
or comment-added events):
|
||||
|
||||
testproject:master:refs/changes/20/420/1^testproject:master:refs/changes/21/421/1"
|
||||
**ZUUL_BRANCH**
|
||||
The target branch for the change that triggered this build
|
||||
**ZUUL_CHANGE**
|
||||
The Gerrit change ID for the change that triggered this build
|
||||
**ZUUL_CHANGE_IDS**
|
||||
All of the Gerrit change IDs that are included in this build (useful
|
||||
when the DependentPipelineManager combines changes for testing)
|
||||
**ZUUL_PATCHSET**
|
||||
The Gerrit patchset number for the change that triggered this build
|
||||
|
||||
The OpenStack project uses the following script to update the
|
||||
repository in a workspace and merge appropriate changes:
|
||||
The following parameters are optional and will only be provided for
|
||||
post-merge (ref-updated) builds:
|
||||
|
||||
https://github.com/openstack/openstack-ci-puppet/blob/master/modules/jenkins/files/slave_scripts/gerrit-git-prep.sh
|
||||
**ZUUL_OLDREV**
|
||||
The SHA1 of the old revision at this ref (recall the ref name is
|
||||
in ZUUL_REF)
|
||||
**ZUUL_NEWREV**
|
||||
The SHA1 of the new revision at this ref (recall the ref name is
|
||||
in ZUUL_REF)
|
||||
**ZUUL_SHORT_OLDREV**
|
||||
The shortened (7 character) SHA1 of the old revision
|
||||
**ZUUL_SHORT_NEWREV**
|
||||
The shortened (7 character) SHA1 of the new revision
|
||||
|
||||
Gerrit events that do not include a change (e.g., ref-updated events
|
||||
which are emitted after a git ref is updated (i.e., a commit is merged
|
||||
to master)) require a slightly different set of parameters:
|
||||
In order to test the correct build, configure the Jenkins Git SCM
|
||||
plugin as follows::
|
||||
|
||||
**UUID**
|
||||
Zuul provided key to link builds with Gerrit events
|
||||
**GERRIT_PROJECT**
|
||||
Zuul provided project name
|
||||
**GERRIT_REFNAME**
|
||||
Zuul provided ref name
|
||||
**GERRIT_OLDREV**
|
||||
Zuul provided old reference for ref-updated
|
||||
**GERRIT_NEWREV**
|
||||
Zuul provided new reference for ref-updated
|
||||
Source Code Management:
|
||||
Git
|
||||
Repositories:
|
||||
Repository URL: <your Gerrit or Zuul repository URL>
|
||||
Advanced:
|
||||
Refspec: ${ZUUL_REF}
|
||||
Branches to build:
|
||||
Branch Specifier: ${ZUUL_COMMIT}
|
||||
Advanced:
|
||||
Clean after checkout: True
|
||||
|
||||
That should be sufficient for a job that only builds a single project.
|
||||
If you have multiple interrelated projects (i.e., they share a Zuul
|
||||
Change Queue) that are built together, you may be able to configure
|
||||
the Git plugin to prepare them, or you may chose to use a shell script
|
||||
instead. The OpenStack project uses the following script to prepare
|
||||
the workspace for its integration testing:
|
||||
|
||||
https://github.com/openstack-ci/devstack-gate/blob/master/devstack-vm-gate-wrap.sh
|
||||
|
|
|
@ -128,11 +128,15 @@ def job_has_changes(*args):
|
|||
path = os.path.join(GIT_ROOT, project)
|
||||
repo = git.Repo(path)
|
||||
ref = job.parameters['ZUUL_REF']
|
||||
sha = job.parameters['ZUUL_COMMIT']
|
||||
repo_messages = [c.message.strip() for c in repo.iter_commits(ref)]
|
||||
repo_shas = [c.hexsha for c in repo.iter_commits(ref)]
|
||||
commit_messages = ['%s-1' % commit.subject for commit in commits]
|
||||
for msg in commit_messages:
|
||||
if msg not in repo_messages:
|
||||
return False
|
||||
if repo_shas[0] != sha:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
|
|
|
@ -219,21 +219,23 @@ class Jenkins(object):
|
|||
dependent_changes = dependent_changes[:]
|
||||
dependent_changes.reverse()
|
||||
uuid = str(uuid4().hex)
|
||||
params = dict(UUID=uuid,
|
||||
GERRIT_PROJECT=change.project.name,
|
||||
params = dict(UUID=uuid, # deprecated
|
||||
ZUUL_UUID=uuid,
|
||||
GERRIT_PROJECT=change.project.name, # deprecated
|
||||
ZUUL_PROJECT=change.project.name)
|
||||
params['ZUUL_PIPELINE'] = pipeline.name
|
||||
if hasattr(change, 'refspec'):
|
||||
changes_str = '^'.join(
|
||||
['%s:%s:%s' % (c.project.name, c.branch, c.refspec)
|
||||
for c in dependent_changes + [change]])
|
||||
params['GERRIT_BRANCH'] = change.branch
|
||||
params['GERRIT_BRANCH'] = change.branch # deprecated
|
||||
params['ZUUL_BRANCH'] = change.branch
|
||||
params['GERRIT_CHANGES'] = changes_str
|
||||
params['GERRIT_CHANGES'] = changes_str # deprecated
|
||||
params['ZUUL_CHANGES'] = changes_str
|
||||
params['ZUUL_REF'] = ('refs/zuul/%s/%s' %
|
||||
(change.branch,
|
||||
change.current_build_set.ref))
|
||||
params['ZUUL_COMMIT'] = change.current_build_set.commit
|
||||
|
||||
zuul_changes = ' '.join(['%s,%s' % (c.number, c.patchset)
|
||||
for c in dependent_changes + [change]])
|
||||
|
@ -241,14 +243,41 @@ class Jenkins(object):
|
|||
params['ZUUL_CHANGE'] = str(change.number)
|
||||
params['ZUUL_PATCHSET'] = str(change.patchset)
|
||||
if hasattr(change, 'ref'):
|
||||
params['GERRIT_REFNAME'] = change.ref
|
||||
params['GERRIT_REFNAME'] = change.ref # deprecated
|
||||
params['ZUUL_REFNAME'] = change.ref
|
||||
params['GERRIT_OLDREV'] = change.oldrev
|
||||
params['GERRIT_OLDREV'] = change.oldrev # deprecated
|
||||
params['ZUUL_OLDREV'] = change.oldrev
|
||||
params['GERRIT_NEWREV'] = change.newrev
|
||||
params['GERRIT_NEWREV'] = change.newrev # deprecated
|
||||
params['ZUUL_NEWREV'] = change.newrev
|
||||
params['ZUUL_SHORT_OLDREV'] = change.oldrev[:7]
|
||||
params['ZUUL_SHORT_NEWREV'] = change.newrev[:7]
|
||||
|
||||
params['ZUUL_REF'] = change.ref
|
||||
params['ZUUL_COMMIT'] = change.newrev
|
||||
|
||||
# This is what we should be heading toward for parameters:
|
||||
|
||||
# required:
|
||||
# ZUUL_UUID
|
||||
# ZUUL_REF (/refs/zuul/..., /refs/tags/foo, master)
|
||||
# ZUUL_COMMIT
|
||||
|
||||
# optional:
|
||||
# ZUUL_PROJECT
|
||||
# ZUUL_PIPELINE
|
||||
|
||||
# optional (changes only):
|
||||
# ZUUL_BRANCH
|
||||
# ZUUL_CHANGE
|
||||
# ZUUL_CHANGE_IDS
|
||||
# ZUUL_PATCHSET
|
||||
|
||||
# optional (ref updated only):
|
||||
# ZUUL_OLDREV
|
||||
# ZUUL_NEWREV
|
||||
# ZUUL_SHORT_NEWREV
|
||||
# ZUUL_SHORT_OLDREV
|
||||
|
||||
if callable(job.parameter_function):
|
||||
job.parameter_function(change, params)
|
||||
self.log.debug("Custom parameter function used for job %s, "
|
||||
|
|
|
@ -81,6 +81,7 @@ class Repo(object):
|
|||
|
||||
def setZuulRef(self, ref, commit):
|
||||
self.repo.refs[ref].commit = commit
|
||||
return self.repo.refs[ref].commit
|
||||
|
||||
def push(self, local, remote):
|
||||
self.log.debug("Pushing %s:%s to %s " % (local, remote,
|
||||
|
@ -112,6 +113,7 @@ class Merger(object):
|
|||
|
||||
def mergeChanges(self, changes, target_ref=None, mode=None):
|
||||
projects = {}
|
||||
commit = None
|
||||
# Reset all repos involved in the change set
|
||||
for change in changes:
|
||||
branches = projects.get(change.project, [])
|
||||
|
@ -148,7 +150,11 @@ class Merger(object):
|
|||
repo.merge(change.refspec)
|
||||
elif mode == model.CHERRY_PICK:
|
||||
repo.cherryPick(change.refspec)
|
||||
repo.setZuulRef(change.branch + '/' + target_ref, 'HEAD')
|
||||
# Keep track of the last commit, it's the commit that
|
||||
# will be passed to jenkins because it's the commit
|
||||
# for the triggering change
|
||||
commit = repo.setZuulRef(change.branch + '/' + target_ref,
|
||||
'HEAD').hexsha
|
||||
except:
|
||||
self.log.info("Unable to merge %s" % change)
|
||||
return False
|
||||
|
@ -169,4 +175,4 @@ class Merger(object):
|
|||
self.log.error("Ref %s did not show up in repo" % ref)
|
||||
return False
|
||||
|
||||
return True
|
||||
return commit
|
||||
|
|
|
@ -402,6 +402,7 @@ class BuildSet(object):
|
|||
self.next_build_set = None
|
||||
self.previous_build_set = None
|
||||
self.ref = None
|
||||
self.commit = None
|
||||
self.unable_to_merge = False
|
||||
|
||||
def setConfiguration(self):
|
||||
|
@ -416,9 +417,6 @@ class BuildSet(object):
|
|||
if not self.ref:
|
||||
self.ref = 'Z' + uuid4().hex
|
||||
|
||||
def getRef(self):
|
||||
return self.ref
|
||||
|
||||
def addBuild(self, build):
|
||||
self.builds[build.job.name] = build
|
||||
build.build_set = self
|
||||
|
|
|
@ -531,18 +531,18 @@ class BasePipelineManager(object):
|
|||
|
||||
def _launchJobs(self, change, jobs):
|
||||
self.log.debug("Launching jobs for change %s" % change)
|
||||
ref = change.current_build_set.getRef()
|
||||
ref = change.current_build_set.ref
|
||||
if hasattr(change, 'refspec') and not ref:
|
||||
change.current_build_set.setConfiguration()
|
||||
ref = change.current_build_set.getRef()
|
||||
ref = change.current_build_set.ref
|
||||
mode = model.MERGE_IF_NECESSARY
|
||||
merged = self.sched.merger.mergeChanges([change], ref, mode=mode)
|
||||
if not merged:
|
||||
commit = self.sched.merger.mergeChanges([change], ref, mode=mode)
|
||||
if not commit:
|
||||
self.log.info("Unable to merge change %s" % change)
|
||||
self.pipeline.setUnableToMerge(change)
|
||||
self.possiblyReportChange(change)
|
||||
return
|
||||
|
||||
change.current_build_set.commit = commit
|
||||
for job in self.pipeline.findJobsToRun(change):
|
||||
self.log.debug("Found job %s for change %s" % (job, change))
|
||||
try:
|
||||
|
@ -956,19 +956,20 @@ class DependentPipelineManager(BasePipelineManager):
|
|||
|
||||
def _launchJobs(self, change, jobs):
|
||||
self.log.debug("Launching jobs for change %s" % change)
|
||||
ref = change.current_build_set.getRef()
|
||||
ref = change.current_build_set.ref
|
||||
if hasattr(change, 'refspec') and not ref:
|
||||
change.current_build_set.setConfiguration()
|
||||
ref = change.current_build_set.getRef()
|
||||
ref = change.current_build_set.ref
|
||||
dependent_changes = self._getDependentChanges(change)
|
||||
dependent_changes.reverse()
|
||||
all_changes = dependent_changes + [change]
|
||||
merged = self.sched.merger.mergeChanges(all_changes, ref)
|
||||
if not merged:
|
||||
commit = self.sched.merger.mergeChanges(all_changes, ref)
|
||||
if not commit:
|
||||
self.log.info("Unable to merge changes %s" % all_changes)
|
||||
self.pipeline.setUnableToMerge(change)
|
||||
self.possiblyReportChange(change)
|
||||
return
|
||||
change.current_build_set.commit = commit
|
||||
#TODO: remove this line after GERRIT_CHANGES is gone
|
||||
dependent_changes = self._getDependentChanges(change)
|
||||
for job in jobs:
|
||||
|
|
Loading…
Reference in New Issue