Zuul rewrite of p_s_r

First draft of an attempt to use zuul to handle the merging rather than
shelling out to git.

Some outstanding issues:

* Zuul swallows git command errors such as merge failures, and doesn't
  even log (except at debug) the fact that they happened. This is a
  problem for a tool we're expecting to use interactively to detect
  merge failures; will need to tweak zuul to make that configurable.

* Zuul's code is lazy: if the local ref it's going to create already
  exists, it continues early on. p_s_r is often run with the same change
  sets (therefor the same target branch name) but new versions, so it's
  imperative that we be less lazy about re-running the merges. Could be
  handled by tweaking zuul to be configurably lazy; or by finding
  something unique (a hash of merger_items perhaps) that is going to
  change any time a new version of any change set is pushed.

* zuul's code doesn't quite do all the things that p_s_r did, so there's
  still a fair bit of shelling out to git in p_s_r. I haven't checked in
  depth to see how many of these things could go away - I suspect that
  some of them might not be needed.

Change-Id: I547869cc88939522212ac923a865a59e2a286242
This commit is contained in:
tchaypo 2014-11-30 15:25:15 +01:00
parent 8921a26674
commit 59d1dc8b40
3 changed files with 29 additions and 10 deletions

View File

@ -2,14 +2,19 @@
import argparse
import json
import logging
import os.path
import re
from subprocess import check_call, check_output
import sys
import yaml
import requests
import zuul.merger.merger
import zuul.model
logging.basicConfig(level=logging.DEBUG)
def normalise_conf(conf):
"""generate full paths etc for easy application later.
@ -49,22 +54,23 @@ def main():
with open(args.refs, 'rt') as arg_file:
CONF = yaml.safe_load(arg_file.read())
CONF = normalise_conf(CONF)
if not os.path.lexists(SRC_ROOT):
os.makedirs(SRC_ROOT)
extra_config = CONF['config']
variables = []
session = requests.Session()
session.headers = {'Accept': 'application/json', 'Accept-Encoding': 'gzip'}
resolved_refs = {}
merger = zuul.merger.merger.Merger(SRC_ROOT,
extra_config['ssh_key'], extra_config['email'],
extra_config['username'])
for repo, (remote, gerrit) in CONF['repos'].items():
if args.repos and repo not in args.repos:
continue
rd = os.path.join(SRC_ROOT, repo)
if not os.path.isdir(os.path.join(rd)):
check_call(['git', 'clone', remote, rd])
git_repo = merger.addProject(repo, remote)
git_repo.update()
refs = CONF['gerrit_refs'].get(repo, ())
@ -113,6 +119,8 @@ def main():
else:
check_call(['git', 'checkout', '-b', branch_name,
'review/master'], cwd=rd)
merge_items = []
for ref in refs:
segments = ref.split('/')
if len(segments) == 3:
@ -121,7 +129,15 @@ def main():
else:
ref = 'refs/changes/%s' % ref
print 'merging in %s' % ref
check_call(['git', 'merge', '--no-edit', ref], cwd=rd)
merge_item = { "number": ref.split('/')[1], "patchset":
ref.split('/')[2], "project": repo, "url": remote,
"branch": branch_name, 'refspec': ref, 'ref': ref,
"merge_mode": zuul.model.MERGER_CHERRY_PICK }
merge_items.append(merge_item)
if merge_items:
merger.mergeChanges(merge_items)
if dirty:
check_call(['git', 'stash', 'pop'], cwd=rd)
normalised_repo = re.sub('[^A-Za-z0-9_]', '_', repo)
@ -140,6 +156,3 @@ def main():
else:
output.write('unset DIB_REPOREF_%s\n' % name)
return 0
sys.exit(main())

View File

@ -1,3 +1,8 @@
config:
ssh_key: "/Users/james/.ssh/identity"
email: "jp@jamezpolley.com"
username: "tchaypo"
# hash of repos to download
repos:
# each hash becomes a path component

View File

@ -1,2 +1,3 @@
PyYAML>=3.11
requests>=2.4.0
-e git+https://git.openstack.org/openstack-infra/zuul#egg=zuul