move the release tools that need to run on secure nodes
Move the scripts that need to run on secure nodes to automatically tag releases from release-tools to project-config Change-Id: Ia451d75998e50c3e82536c6cad72e851e1c8f18e Depends-On: I66e15cd0fa270e9b7a5cb3849cb8d14fa4c617b1 Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This commit is contained in:
parent
53b9c756a5
commit
91577f551a
78
README.rst
78
README.rst
|
@ -26,12 +26,12 @@ if you start a release and run into trouble with launchpad auth.
|
||||||
|
|
||||||
When a release request is ready to be approved, follow these steps:
|
When a release request is ready to be approved, follow these steps:
|
||||||
|
|
||||||
1. The release team member taking responsibility for the
|
#. The release team member taking responsibility for the
|
||||||
release should approve the change in ``openstack/releases``.
|
release should approve the change in ``openstack/releases``.
|
||||||
Release requests should not be approved until we are actually ready
|
Release requests should not be approved until we are actually ready
|
||||||
to cut the release.
|
to cut the release.
|
||||||
|
|
||||||
2. After the release request merges, check out or update a local copy
|
#. After the release request merges, check out or update a local copy
|
||||||
of ``openstack/releases`` to get the new version of the file under
|
of ``openstack/releases`` to get the new version of the file under
|
||||||
the ``deliverables`` directory. Make sure you check out the
|
the ``deliverables`` directory. Make sure you check out the
|
||||||
releases repository to the commit with the new release request you
|
releases repository to the commit with the new release request you
|
||||||
|
@ -39,22 +39,38 @@ When a release request is ready to be approved, follow these steps:
|
||||||
time. The release tools only look at the most recent commit to
|
time. The release tools only look at the most recent commit to
|
||||||
detect which deliverable files have changed.
|
detect which deliverable files have changed.
|
||||||
|
|
||||||
3. In a local copy of this
|
#. Check out or update a local copy of
|
||||||
``openstack-infra/release-tools`` repository, run
|
``openstack-infra/project-config`` repository, which contains the
|
||||||
``release_from_yaml.sh``, giving the path to the
|
tools for tagging a release.
|
||||||
|
|
||||||
|
#. Change directories to
|
||||||
|
``project-config/jenkins/scripts/release-tools``.
|
||||||
|
|
||||||
|
#. Create or update a virtualenv and install all of the dependencies
|
||||||
|
for the release tools using
|
||||||
|
``openstack-infra/project-config/jenkins/scripts/release-tools/requirements.txt``
|
||||||
|
as the basis.
|
||||||
|
|
||||||
|
For example::
|
||||||
|
|
||||||
|
$ virtualenv .venv
|
||||||
|
$ source .venv/bin/activate
|
||||||
|
$ pip install -U -r requirements.txt
|
||||||
|
|
||||||
|
#. Run ``release_from_yaml.sh``, giving the path to the
|
||||||
``openstack/releases`` repository.
|
``openstack/releases`` repository.
|
||||||
|
|
||||||
For example::
|
For example::
|
||||||
|
|
||||||
$ ./release_from_yaml.sh ~/repos/openstack/releases
|
$ ./release_from_yaml.sh ~/repos/openstack/releases
|
||||||
|
|
||||||
4. As the release script runs, it will prompt you for your GPG key
|
#. As the release script runs, it will prompt you for your GPG key
|
||||||
passphrase before adding the tag. This gives you a last chance to
|
passphrase before adding the tag. This gives you a last chance to
|
||||||
review the proposed tag before proceeding. After the tag is created
|
review the proposed tag before proceeding. After the tag is created
|
||||||
locally and pushed up to the remote server, the script will push
|
locally and pushed up to the remote server, the script will push
|
||||||
comments to closed Launchpad bugs since the previous tag.
|
comments to closed Launchpad bugs since the previous tag.
|
||||||
|
|
||||||
5. Announce the release.
|
#. Announce the release.
|
||||||
|
|
||||||
1. Milestones are manually announced once all projects are done
|
1. Milestones are manually announced once all projects are done
|
||||||
(usually at the closing of the milestone window), using an email
|
(usually at the closing of the milestone window), using an email
|
||||||
|
@ -84,51 +100,11 @@ Top-level scripts
|
||||||
|
|
||||||
The top-level scripts call the various base tools to get their work done.
|
The top-level scripts call the various base tools to get their work done.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
release_from_yaml.sh
|
The scripts involved in tagging releases have moved from this
|
||||||
--------------------
|
repository to
|
||||||
|
``openstack-infra/project-config/jenkins/scripts/release-tools``.
|
||||||
This script takes YAML files describing deliverables to release (like those
|
|
||||||
living in ``openstack/releases``) and calls the release.sh script (see below)
|
|
||||||
to apply the corresponding tags. It will create a tag for the last release
|
|
||||||
mentioned in the file(s). You can point it to specific YAML files, or to a
|
|
||||||
local git repository (in which case it will look at the files modified in the
|
|
||||||
most recent commit).
|
|
||||||
|
|
||||||
Examples:
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
./release_from_yaml.sh ../openstack-releases deliverables/mitaka/nova.yaml
|
|
||||||
|
|
||||||
Call release.sh for all repositories mentioned in the last release added
|
|
||||||
to ../openstack-releases/deliverables/mitaka/nova.yaml
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
./release_from_yaml.sh ../openstack-releases
|
|
||||||
|
|
||||||
Look into the git repository at ../openstack-releases for deliverable YAML
|
|
||||||
files modified at the last commit, and call release.sh for all repositories
|
|
||||||
mentioned on the last release in each such file.
|
|
||||||
|
|
||||||
|
|
||||||
release.sh
|
|
||||||
----------
|
|
||||||
|
|
||||||
This script creates a tag on a given repository SHA and pushes it to Gerrit.
|
|
||||||
Additionally it will add a message on Launchpad bugs that are mentioned as
|
|
||||||
"closed" in git commit messages since the last tag on the same series.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
./release.sh openstack/oslo.rootwrap mitaka 3.0.3 gerrit/master
|
|
||||||
|
|
||||||
Apply a 3.0.3 tag (associated to the mitaka series) to the gerrit master
|
|
||||||
HEAD of the openstack/oslo.rootwrap reporitory, and add a comment for each
|
|
||||||
closed bug mentioned in commit messages since the previous mitaka tag (3.0.2).
|
|
||||||
|
|
||||||
announce.sh
|
announce.sh
|
||||||
-----------
|
-----------
|
||||||
|
|
|
@ -1,78 +0,0 @@
|
||||||
#!/usr/bin/env python
|
|
||||||
#
|
|
||||||
# Add a comment on a number of Launchpad bugs
|
|
||||||
#
|
|
||||||
# Copyright 2015 Thierry Carrez <thierry@openstack.org>
|
|
||||||
# All Rights Reserved.
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
||||||
# not use this file except in compliance with the License. You may obtain
|
|
||||||
# a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
||||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
||||||
# License for the specific language governing permissions and limitations
|
|
||||||
# under the License.
|
|
||||||
|
|
||||||
from __future__ import print_function
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import os
|
|
||||||
|
|
||||||
import launchpadlib.launchpad
|
|
||||||
import lazr.restfulclient.errors
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
# Parameters
|
|
||||||
parser = argparse.ArgumentParser(description="Add comment on bugs")
|
|
||||||
parser.add_argument('--subject', help='The comment subject',
|
|
||||||
default='Comment added by add_comment')
|
|
||||||
parser.add_argument('--content', help='The comment content',
|
|
||||||
default='Comment added by add_comment')
|
|
||||||
lp_grp = parser.add_argument_group('launchpad')
|
|
||||||
lp_grp.add_argument(
|
|
||||||
"--test",
|
|
||||||
action='store_const',
|
|
||||||
dest='lp_service_root',
|
|
||||||
const='staging',
|
|
||||||
default='production',
|
|
||||||
help='Use LP staging server to test',
|
|
||||||
)
|
|
||||||
lp_grp.add_argument(
|
|
||||||
'--credentials-file', '-c',
|
|
||||||
dest='lp_creds_file',
|
|
||||||
default=os.environ.get('LP_CREDS_FILE'),
|
|
||||||
help=('plain-text credentials file, '
|
|
||||||
'defaults to value of $LP_CREDS_FILE'),
|
|
||||||
)
|
|
||||||
parser.add_argument('bugs', type=int, nargs='+',
|
|
||||||
help='Bugs to add comment to')
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
# Connect to Launchpad
|
|
||||||
print("Connecting to Launchpad...")
|
|
||||||
launchpad = launchpadlib.launchpad.Launchpad.login_with(
|
|
||||||
application_name='openstack-releasing',
|
|
||||||
service_root=args.lp_service_root,
|
|
||||||
credentials_file=args.lp_creds_file,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Add comment
|
|
||||||
for bugid in args.bugs:
|
|
||||||
print("Adding comment to #%d..." % bugid, end='')
|
|
||||||
try:
|
|
||||||
bug = launchpad.bugs[bugid]
|
|
||||||
bug.newMessage(subject=args.subject, content=args.content)
|
|
||||||
print (" done.")
|
|
||||||
except lazr.restfulclient.errors.ServerError as e:
|
|
||||||
print(" TIMEOUT during save !")
|
|
||||||
except Exception as e:
|
|
||||||
print(" ERROR during save ! (%s)" % e)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
129
release.sh
129
release.sh
|
@ -1,129 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
#
|
|
||||||
# Script to release a project in one shot, including the git tag and
|
|
||||||
# launchpad updates.
|
|
||||||
#
|
|
||||||
# Copyright 2015 Thierry Carrez <thierry@openstack.org>
|
|
||||||
# All Rights Reserved.
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
||||||
# not use this file except in compliance with the License. You may obtain
|
|
||||||
# a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
||||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
||||||
# License for the specific language governing permissions and limitations
|
|
||||||
# under the License.
|
|
||||||
|
|
||||||
set -e
|
|
||||||
|
|
||||||
TOOLSDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
|
||||||
source $TOOLSDIR/functions
|
|
||||||
|
|
||||||
function usage {
|
|
||||||
echo "Usage: release.sh [-a] repository series version SHA announce include_pypi first-full-release extra-metadata"
|
|
||||||
echo
|
|
||||||
echo "Example: release.sh openstack/oslo.rootwrap mitaka 3.0.3 gerrit/master openstack-dev@lists.openstack.org yes no 'meta:release:Workflow+1: Doug Hellmann <doug@doughellmann.com>'"
|
|
||||||
}
|
|
||||||
|
|
||||||
if [ $# -lt 5 ]; then
|
|
||||||
usage
|
|
||||||
exit 2
|
|
||||||
fi
|
|
||||||
|
|
||||||
REPO=$1
|
|
||||||
SERIES=$2
|
|
||||||
VERSION=$3
|
|
||||||
SHA=$4
|
|
||||||
ANNOUNCE=$5
|
|
||||||
INCLUDE_PYPI=${6:-no}
|
|
||||||
FIRST_FULL=${7:-no}
|
|
||||||
EXTRA_METADATA="$8"
|
|
||||||
|
|
||||||
SHORTNAME=`basename $REPO`
|
|
||||||
|
|
||||||
pre_release_pat='\.[[:digit:]]+[ab][[:digit:]]+'
|
|
||||||
rc_release_pat='\.[[:digit:]]+rc[[:digit:]]+'
|
|
||||||
if [[ $VERSION =~ $pre_release_pat ]]; then
|
|
||||||
RELEASETYPE="development milestone"
|
|
||||||
elif [[ $VERSION =~ $rc_release_pat ]]; then
|
|
||||||
RELEASETYPE="release candidate"
|
|
||||||
else
|
|
||||||
RELEASETYPE="release"
|
|
||||||
fi
|
|
||||||
|
|
||||||
setup_temp_space release-tag-$SHORTNAME
|
|
||||||
clone_repo $REPO
|
|
||||||
REPODIR="$(cd $REPO && pwd)"
|
|
||||||
cd $REPODIR
|
|
||||||
TARGETSHA=`git log -1 $SHA --format='%H'`
|
|
||||||
|
|
||||||
# Determine the most recent tag before we add the new one.
|
|
||||||
PREVIOUS=$(get_last_tag $TARGETSHA)
|
|
||||||
|
|
||||||
title "Tagging $TARGETSHA as $VERSION"
|
|
||||||
if git show-ref "$VERSION"; then
|
|
||||||
echo "$REPO already has a version $VERSION tag"
|
|
||||||
# Reset the notion of "previous" to the version associated with
|
|
||||||
# the parent of the commit being tagged, since the tag we're
|
|
||||||
# applying already exists.
|
|
||||||
PREVIOUS=$(get_last_tag ${TARGETSHA}^1)
|
|
||||||
else
|
|
||||||
# WARNING(dhellmann): announce.sh expects to be able to parse this
|
|
||||||
# commit message, so if you change the format you may have to
|
|
||||||
# update announce.sh as well.
|
|
||||||
TAGMSG="$SHORTNAME $VERSION $RELEASETYPE
|
|
||||||
|
|
||||||
meta:version: $VERSION
|
|
||||||
meta:series: $SERIES
|
|
||||||
meta:release-type: $RELEASETYPE
|
|
||||||
meta:announce: $ANNOUNCE
|
|
||||||
meta:pypi: $INCLUDE_PYPI
|
|
||||||
meta:first: $FIRST_FULL
|
|
||||||
$EXTRA_METADATA
|
|
||||||
"
|
|
||||||
echo "Tag message is '$TAGMSG'"
|
|
||||||
git tag -m "$TAGMSG" -s "$VERSION" $TARGETSHA
|
|
||||||
git push gerrit $VERSION
|
|
||||||
fi
|
|
||||||
|
|
||||||
# We don't want to die just because we can't update some bug reports,
|
|
||||||
# so ignore failures.
|
|
||||||
set +e
|
|
||||||
|
|
||||||
title "Adding comments to fixed bugs"
|
|
||||||
BUGS=$(git log $PREVIOUS..$VERSION | egrep -i "Closes(.| )Bug:" | egrep -o "[0-9]+")
|
|
||||||
if [[ -z "$BUGS" ]]; then
|
|
||||||
echo "No bugs found $PREVIOUS .. $VERSION"
|
|
||||||
else
|
|
||||||
$TOOLSDIR/launchpad_add_comment.py \
|
|
||||||
--subject="Fix included in $REPO $VERSION" \
|
|
||||||
--content="This issue was fixed in the $REPO $VERSION $RELEASETYPE." \
|
|
||||||
$BUGS
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Try to propose a constraints update for libraries.
|
|
||||||
if [[ $INCLUDE_PYPI == "yes" ]]; then
|
|
||||||
title "Proposing constraints update"
|
|
||||||
dist_name=$(python setup.py --name)
|
|
||||||
if [[ -z "$dist_name" ]]; then
|
|
||||||
echo "Could not determine the name of the constraint to update"
|
|
||||||
else
|
|
||||||
cd $MYTMPDIR
|
|
||||||
clone_repo openstack/requirements stable/$SERIES
|
|
||||||
cd openstack/requirements
|
|
||||||
git checkout -b "$dist_name-$VERSION"
|
|
||||||
sed -e "s/^${dist_name}=.*/$dist_name===$VERSION/" --in-place upper-constraints.txt
|
|
||||||
git commit -a -m "update constraint for $dist_name to new release $VERSION
|
|
||||||
|
|
||||||
$TAGMSG
|
|
||||||
"
|
|
||||||
git show
|
|
||||||
git review -t 'new-release'
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
exit 0
|
|
|
@ -1,71 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
#
|
|
||||||
# Script to release projects based on changes to the deliverables
|
|
||||||
# files in the openstack/releases repository.
|
|
||||||
#
|
|
||||||
# All Rights Reserved.
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
||||||
# not use this file except in compliance with the License. You may obtain
|
|
||||||
# a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
||||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
||||||
# License for the specific language governing permissions and limitations
|
|
||||||
# under the License.
|
|
||||||
|
|
||||||
set -e
|
|
||||||
|
|
||||||
TOOLSDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
|
||||||
source $TOOLSDIR/functions
|
|
||||||
|
|
||||||
function usage {
|
|
||||||
echo "Usage: release_from_yaml.sh releases_repository [deliverable_files]"
|
|
||||||
echo
|
|
||||||
echo "Example: release_from_yaml.sh ~/repos/openstack/releases"
|
|
||||||
echo "Example: release_from_yaml.sh ~/repos/openstack/releases"
|
|
||||||
echo "Example: release_from_yaml.sh ~/repos/openstack/releases deliverables/mitaka/oslo.config.yaml"
|
|
||||||
}
|
|
||||||
|
|
||||||
if [ $# -lt 1 ]; then
|
|
||||||
echo "ERROR: No releases_repository specified"
|
|
||||||
echo
|
|
||||||
usage
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
RELEASES_REPO="$1"
|
|
||||||
shift
|
|
||||||
DELIVERABLES="$@"
|
|
||||||
|
|
||||||
# Configure git to pull the notes where gerrit stores review history
|
|
||||||
# like who approved a patch.
|
|
||||||
cd $RELEASES_REPO
|
|
||||||
if ! git config --get remote.origin.fetch | grep -q refs/notes/review; then
|
|
||||||
git config --add remote.origin.fetch refs/notes/review:refs/notes/review
|
|
||||||
git config --add core.notesRef refs/notes/review
|
|
||||||
git remote update origin
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Look for metadata about the release instructions to include in the
|
|
||||||
# tag message.
|
|
||||||
if git show --no-patch --pretty=format:%P | grep -q ' '; then
|
|
||||||
# multiple parents, look at the submitted patch instead of the
|
|
||||||
# merge commit
|
|
||||||
parent=$(git show --no-patch --pretty=format:%P | cut -f2 -d' ')
|
|
||||||
else
|
|
||||||
# single parent, look at the current patch
|
|
||||||
parent=''
|
|
||||||
fi
|
|
||||||
RELEASE_META=$(git show --format=full --show-notes=review $parent | egrep -i '(Author|Commit:|Code-Review|Workflow|Change-Id)' | sed -e 's/^ //g' -e 's/^/meta:release:/g')
|
|
||||||
|
|
||||||
$TOOLSDIR/list_deliverable_changes.py -r $RELEASES_REPO $DELIVERABLES \
|
|
||||||
| while read deliverable series version repo hash announce_to pypi first_full; do
|
|
||||||
title "$repo $series $version $hash $announce_to"
|
|
||||||
$TOOLSDIR/release.sh $repo $series $version $hash $announce_to $pypi $first_full "$RELEASE_META"
|
|
||||||
done
|
|
||||||
|
|
||||||
exit 0
|
|
Loading…
Reference in New Issue