move the release tools that need to run on secure nodes
Move the scripts that need to run on secure nodes to automatically tag releases from release-tools to project-config Change-Id: Ia451d75998e50c3e82536c6cad72e851e1c8f18e Depends-On: I66e15cd0fa270e9b7a5cb3849cb8d14fa4c617b1 Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This commit is contained in:
Doug Hellmann
parent
53b9c756a5
commit
91577f551a
78
README.rst
78
README.rst
|
|
@ -26,12 +26,12 @@ if you start a release and run into trouble with launchpad auth.
|
|||
|
||||
When a release request is ready to be approved, follow these steps:
|
||||
|
||||
1. The release team member taking responsibility for the
|
||||
#. The release team member taking responsibility for the
|
||||
release should approve the change in ``openstack/releases``.
|
||||
Release requests should not be approved until we are actually ready
|
||||
to cut the release.
|
||||
|
||||
2. After the release request merges, check out or update a local copy
|
||||
#. After the release request merges, check out or update a local copy
|
||||
of ``openstack/releases`` to get the new version of the file under
|
||||
the ``deliverables`` directory. Make sure you check out the
|
||||
releases repository to the commit with the new release request you
|
||||
|
|
@ -39,22 +39,38 @@ When a release request is ready to be approved, follow these steps:
|
|||
time. The release tools only look at the most recent commit to
|
||||
detect which deliverable files have changed.
|
||||
|
||||
3. In a local copy of this
|
||||
``openstack-infra/release-tools`` repository, run
|
||||
``release_from_yaml.sh``, giving the path to the
|
||||
#. Check out or update a local copy of
|
||||
``openstack-infra/project-config`` repository, which contains the
|
||||
tools for tagging a release.
|
||||
|
||||
#. Change directories to
|
||||
``project-config/jenkins/scripts/release-tools``.
|
||||
|
||||
#. Create or update a virtualenv and install all of the dependencies
|
||||
for the release tools using
|
||||
``openstack-infra/project-config/jenkins/scripts/release-tools/requirements.txt``
|
||||
as the basis.
|
||||
|
||||
For example::
|
||||
|
||||
$ virtualenv .venv
|
||||
$ source .venv/bin/activate
|
||||
$ pip install -U -r requirements.txt
|
||||
|
||||
#. Run ``release_from_yaml.sh``, giving the path to the
|
||||
``openstack/releases`` repository.
|
||||
|
||||
For example::
|
||||
|
||||
$ ./release_from_yaml.sh ~/repos/openstack/releases
|
||||
|
||||
4. As the release script runs, it will prompt you for your GPG key
|
||||
#. As the release script runs, it will prompt you for your GPG key
|
||||
passphrase before adding the tag. This gives you a last chance to
|
||||
review the proposed tag before proceeding. After the tag is created
|
||||
locally and pushed up to the remote server, the script will push
|
||||
comments to closed Launchpad bugs since the previous tag.
|
||||
|
||||
5. Announce the release.
|
||||
#. Announce the release.
|
||||
|
||||
1. Milestones are manually announced once all projects are done
|
||||
(usually at the closing of the milestone window), using an email
|
||||
|
|
@ -84,51 +100,11 @@ Top-level scripts
|
|||
|
||||
The top-level scripts call the various base tools to get their work done.
|
||||
|
||||
.. note::
|
||||
|
||||
release_from_yaml.sh
|
||||
--------------------
|
||||
|
||||
This script takes YAML files describing deliverables to release (like those
|
||||
living in ``openstack/releases``) and calls the release.sh script (see below)
|
||||
to apply the corresponding tags. It will create a tag for the last release
|
||||
mentioned in the file(s). You can point it to specific YAML files, or to a
|
||||
local git repository (in which case it will look at the files modified in the
|
||||
most recent commit).
|
||||
|
||||
Examples:
|
||||
|
||||
::
|
||||
|
||||
./release_from_yaml.sh ../openstack-releases deliverables/mitaka/nova.yaml
|
||||
|
||||
Call release.sh for all repositories mentioned in the last release added
|
||||
to ../openstack-releases/deliverables/mitaka/nova.yaml
|
||||
|
||||
::
|
||||
|
||||
./release_from_yaml.sh ../openstack-releases
|
||||
|
||||
Look into the git repository at ../openstack-releases for deliverable YAML
|
||||
files modified at the last commit, and call release.sh for all repositories
|
||||
mentioned on the last release in each such file.
|
||||
|
||||
|
||||
release.sh
|
||||
----------
|
||||
|
||||
This script creates a tag on a given repository SHA and pushes it to Gerrit.
|
||||
Additionally it will add a message on Launchpad bugs that are mentioned as
|
||||
"closed" in git commit messages since the last tag on the same series.
|
||||
|
||||
Example:
|
||||
|
||||
::
|
||||
|
||||
./release.sh openstack/oslo.rootwrap mitaka 3.0.3 gerrit/master
|
||||
|
||||
Apply a 3.0.3 tag (associated to the mitaka series) to the gerrit master
|
||||
HEAD of the openstack/oslo.rootwrap reporitory, and add a comment for each
|
||||
closed bug mentioned in commit messages since the previous mitaka tag (3.0.2).
|
||||
The scripts involved in tagging releases have moved from this
|
||||
repository to
|
||||
``openstack-infra/project-config/jenkins/scripts/release-tools``.
|
||||
|
||||
announce.sh
|
||||
-----------
|
||||
|
|
|
|||
|
|
@ -1,78 +0,0 @@
|
|||
#!/usr/bin/env python
|
||||
#
|
||||
# Add a comment on a number of Launchpad bugs
|
||||
#
|
||||
# Copyright 2015 Thierry Carrez <thierry@openstack.org>
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import argparse
|
||||
import os
|
||||
|
||||
import launchpadlib.launchpad
|
||||
import lazr.restfulclient.errors
|
||||
|
||||
|
||||
def main():
|
||||
# Parameters
|
||||
parser = argparse.ArgumentParser(description="Add comment on bugs")
|
||||
parser.add_argument('--subject', help='The comment subject',
|
||||
default='Comment added by add_comment')
|
||||
parser.add_argument('--content', help='The comment content',
|
||||
default='Comment added by add_comment')
|
||||
lp_grp = parser.add_argument_group('launchpad')
|
||||
lp_grp.add_argument(
|
||||
"--test",
|
||||
action='store_const',
|
||||
dest='lp_service_root',
|
||||
const='staging',
|
||||
default='production',
|
||||
help='Use LP staging server to test',
|
||||
)
|
||||
lp_grp.add_argument(
|
||||
'--credentials-file', '-c',
|
||||
dest='lp_creds_file',
|
||||
default=os.environ.get('LP_CREDS_FILE'),
|
||||
help=('plain-text credentials file, '
|
||||
'defaults to value of $LP_CREDS_FILE'),
|
||||
)
|
||||
parser.add_argument('bugs', type=int, nargs='+',
|
||||
help='Bugs to add comment to')
|
||||
args = parser.parse_args()
|
||||
|
||||
# Connect to Launchpad
|
||||
print("Connecting to Launchpad...")
|
||||
launchpad = launchpadlib.launchpad.Launchpad.login_with(
|
||||
application_name='openstack-releasing',
|
||||
service_root=args.lp_service_root,
|
||||
credentials_file=args.lp_creds_file,
|
||||
)
|
||||
|
||||
# Add comment
|
||||
for bugid in args.bugs:
|
||||
print("Adding comment to #%d..." % bugid, end='')
|
||||
try:
|
||||
bug = launchpad.bugs[bugid]
|
||||
bug.newMessage(subject=args.subject, content=args.content)
|
||||
print (" done.")
|
||||
except lazr.restfulclient.errors.ServerError as e:
|
||||
print(" TIMEOUT during save !")
|
||||
except Exception as e:
|
||||
print(" ERROR during save ! (%s)" % e)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
129
release.sh
129
release.sh
|
|
@ -1,129 +0,0 @@
|
|||
#!/bin/bash
|
||||
#
|
||||
# Script to release a project in one shot, including the git tag and
|
||||
# launchpad updates.
|
||||
#
|
||||
# Copyright 2015 Thierry Carrez <thierry@openstack.org>
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
set -e
|
||||
|
||||
TOOLSDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
source $TOOLSDIR/functions
|
||||
|
||||
function usage {
|
||||
echo "Usage: release.sh [-a] repository series version SHA announce include_pypi first-full-release extra-metadata"
|
||||
echo
|
||||
echo "Example: release.sh openstack/oslo.rootwrap mitaka 3.0.3 gerrit/master openstack-dev@lists.openstack.org yes no 'meta:release:Workflow+1: Doug Hellmann <doug@doughellmann.com>'"
|
||||
}
|
||||
|
||||
if [ $# -lt 5 ]; then
|
||||
usage
|
||||
exit 2
|
||||
fi
|
||||
|
||||
REPO=$1
|
||||
SERIES=$2
|
||||
VERSION=$3
|
||||
SHA=$4
|
||||
ANNOUNCE=$5
|
||||
INCLUDE_PYPI=${6:-no}
|
||||
FIRST_FULL=${7:-no}
|
||||
EXTRA_METADATA="$8"
|
||||
|
||||
SHORTNAME=`basename $REPO`
|
||||
|
||||
pre_release_pat='\.[[:digit:]]+[ab][[:digit:]]+'
|
||||
rc_release_pat='\.[[:digit:]]+rc[[:digit:]]+'
|
||||
if [[ $VERSION =~ $pre_release_pat ]]; then
|
||||
RELEASETYPE="development milestone"
|
||||
elif [[ $VERSION =~ $rc_release_pat ]]; then
|
||||
RELEASETYPE="release candidate"
|
||||
else
|
||||
RELEASETYPE="release"
|
||||
fi
|
||||
|
||||
setup_temp_space release-tag-$SHORTNAME
|
||||
clone_repo $REPO
|
||||
REPODIR="$(cd $REPO && pwd)"
|
||||
cd $REPODIR
|
||||
TARGETSHA=`git log -1 $SHA --format='%H'`
|
||||
|
||||
# Determine the most recent tag before we add the new one.
|
||||
PREVIOUS=$(get_last_tag $TARGETSHA)
|
||||
|
||||
title "Tagging $TARGETSHA as $VERSION"
|
||||
if git show-ref "$VERSION"; then
|
||||
echo "$REPO already has a version $VERSION tag"
|
||||
# Reset the notion of "previous" to the version associated with
|
||||
# the parent of the commit being tagged, since the tag we're
|
||||
# applying already exists.
|
||||
PREVIOUS=$(get_last_tag ${TARGETSHA}^1)
|
||||
else
|
||||
# WARNING(dhellmann): announce.sh expects to be able to parse this
|
||||
# commit message, so if you change the format you may have to
|
||||
# update announce.sh as well.
|
||||
TAGMSG="$SHORTNAME $VERSION $RELEASETYPE
|
||||
|
||||
meta:version: $VERSION
|
||||
meta:series: $SERIES
|
||||
meta:release-type: $RELEASETYPE
|
||||
meta:announce: $ANNOUNCE
|
||||
meta:pypi: $INCLUDE_PYPI
|
||||
meta:first: $FIRST_FULL
|
||||
$EXTRA_METADATA
|
||||
"
|
||||
echo "Tag message is '$TAGMSG'"
|
||||
git tag -m "$TAGMSG" -s "$VERSION" $TARGETSHA
|
||||
git push gerrit $VERSION
|
||||
fi
|
||||
|
||||
# We don't want to die just because we can't update some bug reports,
|
||||
# so ignore failures.
|
||||
set +e
|
||||
|
||||
title "Adding comments to fixed bugs"
|
||||
BUGS=$(git log $PREVIOUS..$VERSION | egrep -i "Closes(.| )Bug:" | egrep -o "[0-9]+")
|
||||
if [[ -z "$BUGS" ]]; then
|
||||
echo "No bugs found $PREVIOUS .. $VERSION"
|
||||
else
|
||||
$TOOLSDIR/launchpad_add_comment.py \
|
||||
--subject="Fix included in $REPO $VERSION" \
|
||||
--content="This issue was fixed in the $REPO $VERSION $RELEASETYPE." \
|
||||
$BUGS
|
||||
fi
|
||||
|
||||
# Try to propose a constraints update for libraries.
|
||||
if [[ $INCLUDE_PYPI == "yes" ]]; then
|
||||
title "Proposing constraints update"
|
||||
dist_name=$(python setup.py --name)
|
||||
if [[ -z "$dist_name" ]]; then
|
||||
echo "Could not determine the name of the constraint to update"
|
||||
else
|
||||
cd $MYTMPDIR
|
||||
clone_repo openstack/requirements stable/$SERIES
|
||||
cd openstack/requirements
|
||||
git checkout -b "$dist_name-$VERSION"
|
||||
sed -e "s/^${dist_name}=.*/$dist_name===$VERSION/" --in-place upper-constraints.txt
|
||||
git commit -a -m "update constraint for $dist_name to new release $VERSION
|
||||
|
||||
$TAGMSG
|
||||
"
|
||||
git show
|
||||
git review -t 'new-release'
|
||||
fi
|
||||
fi
|
||||
|
||||
exit 0
|
||||
|
|
@ -1,71 +0,0 @@
|
|||
#!/bin/bash
|
||||
#
|
||||
# Script to release projects based on changes to the deliverables
|
||||
# files in the openstack/releases repository.
|
||||
#
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
set -e
|
||||
|
||||
TOOLSDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
source $TOOLSDIR/functions
|
||||
|
||||
function usage {
|
||||
echo "Usage: release_from_yaml.sh releases_repository [deliverable_files]"
|
||||
echo
|
||||
echo "Example: release_from_yaml.sh ~/repos/openstack/releases"
|
||||
echo "Example: release_from_yaml.sh ~/repos/openstack/releases"
|
||||
echo "Example: release_from_yaml.sh ~/repos/openstack/releases deliverables/mitaka/oslo.config.yaml"
|
||||
}
|
||||
|
||||
if [ $# -lt 1 ]; then
|
||||
echo "ERROR: No releases_repository specified"
|
||||
echo
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
RELEASES_REPO="$1"
|
||||
shift
|
||||
DELIVERABLES="$@"
|
||||
|
||||
# Configure git to pull the notes where gerrit stores review history
|
||||
# like who approved a patch.
|
||||
cd $RELEASES_REPO
|
||||
if ! git config --get remote.origin.fetch | grep -q refs/notes/review; then
|
||||
git config --add remote.origin.fetch refs/notes/review:refs/notes/review
|
||||
git config --add core.notesRef refs/notes/review
|
||||
git remote update origin
|
||||
fi
|
||||
|
||||
# Look for metadata about the release instructions to include in the
|
||||
# tag message.
|
||||
if git show --no-patch --pretty=format:%P | grep -q ' '; then
|
||||
# multiple parents, look at the submitted patch instead of the
|
||||
# merge commit
|
||||
parent=$(git show --no-patch --pretty=format:%P | cut -f2 -d' ')
|
||||
else
|
||||
# single parent, look at the current patch
|
||||
parent=''
|
||||
fi
|
||||
RELEASE_META=$(git show --format=full --show-notes=review $parent | egrep -i '(Author|Commit:|Code-Review|Workflow|Change-Id)' | sed -e 's/^ //g' -e 's/^/meta:release:/g')
|
||||
|
||||
$TOOLSDIR/list_deliverable_changes.py -r $RELEASES_REPO $DELIVERABLES \
|
||||
| while read deliverable series version repo hash announce_to pypi first_full; do
|
||||
title "$repo $series $version $hash $announce_to"
|
||||
$TOOLSDIR/release.sh $repo $series $version $hash $announce_to $pypi $first_full "$RELEASE_META"
|
||||
done
|
||||
|
||||
exit 0
|
||||
Loading…
Reference in New Issue