Support multiple triggers

Add the ability for Zuul to accept inputs from multiple trigger
sources simultaneously.

Pipelines are associated with exactly one trigger, which must now
be named in the configuration file.

Co-Authored-By: Monty Taylor <mordred@inaugust.com>

Change-Id: Ief2b31a7b8d85d30817f2747c1e2635f71ea24b9
This commit is contained in:
James E. Blair 2013-07-29 17:06:47 -07:00
parent 3cd25034b6
commit 6c358e72ea
20 changed files with 214 additions and 147 deletions

View File

@ -11,13 +11,19 @@ Since 1.3.0:
QueueItem has the full context for why the change is being run QueueItem has the full context for why the change is being run
(including the pipeline, items ahead and behind, etc.). The Change (including the pipeline, items ahead and behind, etc.). The Change
is still available via the "change" attribute on the QueueItem. The is still available via the "change" attribute on the QueueItem. The
second argument is now the Job that is about to be run, ande the second argument is now the Job that is about to be run, and the
parameter dictionary is shifted to the third position. parameter dictionary is shifted to the third position.
* The ZUUL_SHORT_* parameters have been removed (the same * The ZUUL_SHORT_* parameters have been removed (the same
functionality may be achieved with a custom parameter function that functionality may be achieved with a custom parameter function that
matches all jobs). matches all jobs).
* Multiple triggers are now supported, in principle (though only
Gerrit is defined currently). Your layout.yaml file will need to
change to add the key "gerrit:" inside of the "triggers:" list to
specify a Gerrit trigger (and facilitate adding other kinds of
triggers later). See the sample layout.yaml.
* The default behavior is now to immediately dequeue changes that have * The default behavior is now to immediately dequeue changes that have
merge conflicts, even those not at the head of the queue. To enable merge conflicts, even those not at the head of the queue. To enable
the old behavior (which would wait until the conflicting change was the old behavior (which would wait until the conflicting change was

View File

@ -67,6 +67,19 @@ List Failing Tests
.tox/py27/bin/activate .tox/py27/bin/activate
testr failing --list testr failing --list
Hanging Tests
-------------
The following will run each test in turn and print the name of the
test as it is run::
. .tox/py27/bin/activate
testr run --subunit | subunit2pyunit
You can compare the output of that to::
python -m testtools.run discover --list
Need More Info? Need More Info?
--------------- ---------------

View File

@ -178,7 +178,8 @@ explanation of each of the parameters::
- name: check - name: check
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
- event: patchset-created
success: success:
verified: 1 verified: 1
failure: failure:
@ -253,51 +254,55 @@ explanation of each of the parameters::
DependentPipelineManager, see: :doc:`gating`. DependentPipelineManager, see: :doc:`gating`.
**trigger** **trigger**
This describes what Gerrit events should be placed in the pipeline. Exactly one trigger source must be supplied for each pipeline.
Triggers are not exclusive -- matching events may be placed in Triggers are not exclusive -- matching events may be placed in
multiple pipelines, and they will behave independently in each of the multiple pipelines, and they will behave independently in each of
pipelines they match. Multiple triggers may be listed. Further the pipelines they match. You may select from the following:
parameters describe the kind of events that match:
*event* **gerrit**
The event name from gerrit. Examples: ``patchset-created``, This describes what Gerrit events should be placed in the
``comment-added``, ``ref-updated``. This field is treated as a pipeline. Multiple gerrit triggers may be listed. Further
regular expression. parameters describe the kind of events that match:
*branch* *event*
The branch associated with the event. Example: ``master``. This The event name from gerrit. Examples: ``patchset-created``,
field is treated as a regular expression, and multiple branches may ``comment-added``, ``ref-updated``. This field is treated as a
be listed. regular expression.
*ref* *branch*
On ref-updated events, the branch parameter is not used, instead the The branch associated with the event. Example: ``master``. This
ref is provided. Currently Gerrit has the somewhat idiosyncratic field is treated as a regular expression, and multiple branches may
behavior of specifying bare refs for branch names (e.g., ``master``), be listed.
but full ref names for other kinds of refs (e.g., ``refs/tags/foo``).
Zuul matches what you put here exactly against what Gerrit
provides. This field is treated as a regular expression, and
multiple refs may be listed.
*approval* *ref*
This is only used for ``comment-added`` events. It only matches if On ref-updated events, the branch parameter is not used, instead the
the event has a matching approval associated with it. Example: ref is provided. Currently Gerrit has the somewhat idiosyncratic
``code-review: 2`` matches a ``+2`` vote on the code review category. behavior of specifying bare refs for branch names (e.g., ``master``),
Multiple approvals may be listed. but full ref names for other kinds of refs (e.g., ``refs/tags/foo``).
Zuul matches what you put here exactly against what Gerrit
provides. This field is treated as a regular expression, and
multiple refs may be listed.
*email_filter* *approval*
This is used for any event. It takes a regex applied on the performer This is only used for ``comment-added`` events. It only matches if
email, i.e Gerrit account email address. If you want to specify the event has a matching approval associated with it. Example:
several email filters, you must use a YAML list. Make sure to use non ``code-review: 2`` matches a ``+2`` vote on the code review category.
greedy matchers and to escapes dots! Multiple approvals may be listed.
Example: ``email_filter: ^.*?@example\.org$``.
*comment_filter* *email_filter*
This is only used for ``comment-added`` events. It accepts a list of This is used for any event. It takes a regex applied on the performer
regexes that are searched for in the comment string. If any of these email, i.e Gerrit account email address. If you want to specify
regexes matches a portion of the comment string the trigger is several email filters, you must use a YAML list. Make sure to use non
matched. ``comment_filter: retrigger`` will match when comments greedy matchers and to escapes dots!
containing 'retrigger' somewhere in the comment text are added to a Example: ``email_filter: ^.*?@example\.org$``.
change.
*comment_filter*
This is only used for ``comment-added`` events. It accepts a list of
regexes that are searched for in the comment string. If any of these
regexes matches a portion of the comment string the trigger is
matched. ``comment_filter: retrigger`` will match when comments
containing 'retrigger' somewhere in the comment text are added to a
change.
**dequeue-on-new-patchset** **dequeue-on-new-patchset**
Normally, if a new patchset is uploaded to a change that is in a Normally, if a new patchset is uploaded to a change that is in a

View File

@ -2,7 +2,8 @@ pipelines:
- name: check - name: check
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
- event: patchset-created
success: success:
verified: 1 verified: 1
failure: failure:
@ -11,8 +12,9 @@ pipelines:
- name: tests - name: tests
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
email_filter: ^.*@example.org$ - event: patchset-created
email_filter: ^.*@example.org$
success: success:
verified: 1 verified: 1
failure: failure:
@ -21,15 +23,17 @@ pipelines:
- name: post - name: post
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: ref-updated gerrit:
ref: ^(?!refs/).*$ - event: ref-updated
ref: ^(?!refs/).*$
- name: gate - name: gate
manager: DependentPipelineManager manager: DependentPipelineManager
trigger: trigger:
- event: comment-added gerrit:
approval: - event: comment-added
- approved: 1 approval:
- approved: 1
start: start:
verified: 0 verified: 0
success: success:

View File

@ -2,7 +2,8 @@ pipelines:
- name: check - name: check
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
- event: patchset-created
success: success:
verified: 1 verified: 1
failure: failure:
@ -11,16 +12,18 @@ pipelines:
- name: post - name: post
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: ref-updated gerrit:
ref: ^(?!refs/).*$ - event: ref-updated
ref: ^(?!refs/).*$
- name: gate - name: gate
manager: DependentPipelineManager manager: DependentPipelineManager
failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures
trigger: trigger:
- event: comment-added gerrit:
approval: - event: comment-added
- approved: 1 approval:
- approved: 1
success: success:
verified: 2 verified: 2
submit: true submit: true

View File

@ -5,7 +5,8 @@ pipelines:
- name: check - name: check
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
- event: patchset-created
success: success:
verified: 1 verified: 1
failure: failure:
@ -14,16 +15,18 @@ pipelines:
- name: post - name: post
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: ref-updated gerrit:
ref: ^(?!refs/).*$ - event: ref-updated
ref: ^(?!refs/).*$
- name: gate - name: gate
manager: DependentPipelineManager manager: DependentPipelineManager
failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures
trigger: trigger:
- event: comment-added gerrit:
approval: - event: comment-added
- approved: 1 approval:
- approved: 1
success: success:
verified: 2 verified: 2
submit: true submit: true
@ -37,14 +40,16 @@ pipelines:
manager: IndependentPipelineManager manager: IndependentPipelineManager
dequeue-on-new-patchset: false dequeue-on-new-patchset: false
trigger: trigger:
- event: comment-added gerrit:
approval: - event: comment-added
- approved: 1 approval:
- approved: 1
- name: dup1 - name: dup1
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: change-restored gerrit:
- event: change-restored
success: success:
verified: 1 verified: 1
failure: failure:
@ -53,7 +58,8 @@ pipelines:
- name: dup2 - name: dup2
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: change-restored gerrit:
- event: change-restored
success: success:
verified: 1 verified: 1
failure: failure:
@ -64,9 +70,10 @@ pipelines:
manager: DependentPipelineManager manager: DependentPipelineManager
failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures
trigger: trigger:
- event: comment-added gerrit:
approval: - event: comment-added
- approved: 1 approval:
- approved: 1
success: success:
verified: 2 verified: 2
submit: true submit: true

View File

@ -2,7 +2,8 @@ pipelines:
- name: check - name: check
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: non-event gerrit:
- event: non-event
projects: projects:
- name: foo - name: foo

View File

@ -2,8 +2,9 @@ pipelines:
- name: check - name: check
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- approval: gerrit:
- approved: 1 - approval:
- approved: 1
projects: projects:
- name: foo - name: foo

View File

@ -2,8 +2,9 @@ pipelines:
- name: check - name: check
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: comment-added gerrit:
approved: 1 - event: comment-added
approved: 1
projects: projects:
- name: foo - name: foo

View File

@ -4,7 +4,8 @@ pipelines:
- name: 'check' - name: 'check'
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
- event: patchset-created
project-templates: project-templates:
- name: template-generic - name: template-generic

View File

@ -4,7 +4,8 @@ pipelines:
- name: 'check' - name: 'check'
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
- event: patchset-created
project-templates: project-templates:
- name: template-generic - name: template-generic

View File

@ -5,7 +5,8 @@ pipelines:
- name: check - name: check
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
- event: patchset-created
success: success:
verified: 1 verified: 1
failure: failure:
@ -14,17 +15,19 @@ pipelines:
- name: post - name: post
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: ref-updated gerrit:
ref: ^(?!refs/).*$ - event: ref-updated
ref: ^(?!refs/).*$
- name: gate - name: gate
manager: DependentPipelineManager manager: DependentPipelineManager
success-message: Your change is awesome. success-message: Your change is awesome.
failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures failure-message: Build failed. For information on how to proceed, see http://wiki.example.org/Test_Failures
trigger: trigger:
- event: comment-added gerrit:
approval: - event: comment-added
- approved: 1 approval:
- approved: 1
success: success:
verified: 2 verified: 2
code-review: 1 code-review: 1

View File

@ -2,7 +2,8 @@ pipelines:
- name: 'check' - name: 'check'
manager: IndependentPipelineManager manager: IndependentPipelineManager
trigger: trigger:
- event: patchset-created gerrit:
- event: patchset-created
project-templates: project-templates:
- name: template-generic - name: template-generic

View File

@ -28,7 +28,7 @@ FIXTURE_DIR = os.path.join(os.path.dirname(__file__),
LAYOUT_RE = re.compile(r'^(good|bad)_.*\.yaml$') LAYOUT_RE = re.compile(r'^(good|bad)_.*\.yaml$')
class testScheduler(testtools.TestCase): class TestLayoutValidator(testtools.TestCase):
def test_layouts(self): def test_layouts(self):
"""Test layout file validation""" """Test layout file validation"""
print print

View File

@ -761,7 +761,7 @@ class TestScheduler(testtools.TestCase):
self.webapp = zuul.webapp.WebApp(self.sched, port=0) self.webapp = zuul.webapp.WebApp(self.sched, port=0)
self.sched.setLauncher(self.launcher) self.sched.setLauncher(self.launcher)
self.sched.setTrigger(self.gerrit) self.sched.registerTrigger(self.gerrit)
self.sched.start() self.sched.start()
self.sched.reconfigure(self.config) self.sched.reconfigure(self.config)
@ -777,7 +777,7 @@ class TestScheduler(testtools.TestCase):
def assertFinalState(self): def assertFinalState(self):
# Make sure that the change cache is cleared # Make sure that the change cache is cleared
self.assertEqual(len(self.sched.trigger._change_cache.keys()), 0) self.assertEqual(len(self.gerrit._change_cache.keys()), 0)
self.assertEmptyQueues() self.assertEmptyQueues()
def shutdown(self): def shutdown(self):
@ -1440,9 +1440,9 @@ class TestScheduler(testtools.TestCase):
self.fake_gerrit.addEvent(B.addApproval('APRV', 1)) self.fake_gerrit.addEvent(B.addApproval('APRV', 1))
self.waitUntilSettled() self.waitUntilSettled()
self.log.debug("len %s " % self.sched.trigger._change_cache.keys()) self.log.debug("len %s " % self.gerrit._change_cache.keys())
# there should still be changes in the cache # there should still be changes in the cache
self.assertNotEqual(len(self.sched.trigger._change_cache.keys()), 0) self.assertNotEqual(len(self.gerrit._change_cache.keys()), 0)
self.worker.hold_jobs_in_build = False self.worker.hold_jobs_in_build = False
self.worker.release() self.worker.release()
@ -1457,21 +1457,19 @@ class TestScheduler(testtools.TestCase):
"Test whether a change is ready to merge" "Test whether a change is ready to merge"
# TODO: move to test_gerrit (this is a unit test!) # TODO: move to test_gerrit (this is a unit test!)
A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A') A = self.fake_gerrit.addFakeChange('org/project', 'master', 'A')
a = self.sched.trigger.getChange(1, 2) trigger = self.sched.layout.pipelines['gate'].trigger
a = self.sched.triggers['gerrit'].getChange(1, 2)
mgr = self.sched.layout.pipelines['gate'].manager mgr = self.sched.layout.pipelines['gate'].manager
self.assertFalse( self.assertFalse(trigger.canMerge(a, mgr.getSubmitAllowNeeds()))
self.sched.trigger.canMerge(a, mgr.getSubmitAllowNeeds()))
A.addApproval('CRVW', 2) A.addApproval('CRVW', 2)
a = self.sched.trigger.getChange(1, 2, refresh=True) a = trigger.getChange(1, 2, refresh=True)
self.assertFalse( self.assertFalse(trigger.canMerge(a, mgr.getSubmitAllowNeeds()))
self.sched.trigger.canMerge(a, mgr.getSubmitAllowNeeds()))
A.addApproval('APRV', 1) A.addApproval('APRV', 1)
a = self.sched.trigger.getChange(1, 2, refresh=True) a = trigger.getChange(1, 2, refresh=True)
self.assertTrue( self.assertTrue(trigger.canMerge(a, mgr.getSubmitAllowNeeds()))
self.sched.trigger.canMerge(a, mgr.getSubmitAllowNeeds())) trigger.maintainCache([])
self.sched.trigger.maintainCache([])
def test_build_configuration(self): def test_build_configuration(self):
"Test that zuul merges the right commits for testing" "Test that zuul merges the right commits for testing"
@ -2347,6 +2345,7 @@ class TestScheduler(testtools.TestCase):
def test_test_config(self): def test_test_config(self):
"Test that we can test the config" "Test that we can test the config"
sched = zuul.scheduler.Scheduler() sched = zuul.scheduler.Scheduler()
sched.registerTrigger(None, 'gerrit')
sched.testConfig(CONFIG.get('zuul', 'layout_config')) sched.testConfig(CONFIG.get('zuul', 'layout_config'))
def test_build_description(self): def test_build_description(self):

View File

@ -105,6 +105,7 @@ class Server(object):
logging.basicConfig(level=logging.DEBUG) logging.basicConfig(level=logging.DEBUG)
self.sched = zuul.scheduler.Scheduler() self.sched = zuul.scheduler.Scheduler()
self.sched.registerTrigger(None, 'gerrit')
layout = self.sched.testConfig(self.config.get('zuul', layout = self.sched.testConfig(self.config.get('zuul',
'layout_config')) 'layout_config'))
if not job_list_path: if not job_list_path:
@ -165,7 +166,7 @@ class Server(object):
webapp = zuul.webapp.WebApp(self.sched) webapp = zuul.webapp.WebApp(self.sched)
self.sched.setLauncher(gearman) self.sched.setLauncher(gearman)
self.sched.setTrigger(gerrit) self.sched.registerTrigger(gerrit)
self.sched.start() self.sched.start()
self.sched.reconfigure(self.config) self.sched.reconfigure(self.config)

View File

@ -35,18 +35,21 @@ class LayoutSchema(object):
variable_dict = v.Schema({}, extra=True) variable_dict = v.Schema({}, extra=True)
trigger = {v.Required('event'): toList(v.Any('patchset-created', gerrit_trigger = {v.Required('event'):
'change-abandoned', toList(v.Any('patchset-created',
'change-restored', 'change-abandoned',
'change-merged', 'change-restored',
'comment-added', 'change-merged',
'ref-updated')), 'comment-added',
'comment_filter': toList(str), 'ref-updated')),
'email_filter': toList(str), 'comment_filter': toList(str),
'branch': toList(str), 'email_filter': toList(str),
'ref': toList(str), 'branch': toList(str),
'approval': toList(variable_dict), 'ref': toList(str),
} 'approval': toList(variable_dict),
}
trigger = v.Required(v.Any({'gerrit': toList(gerrit_trigger)}))
pipeline = {v.Required('name'): str, pipeline = {v.Required('name'): str,
v.Required('manager'): manager, v.Required('manager'): manager,
@ -56,7 +59,7 @@ class LayoutSchema(object):
'failure-message': str, 'failure-message': str,
'dequeue-on-new-patchset': bool, 'dequeue-on-new-patchset': bool,
'dequeue-on-conflict': bool, 'dequeue-on-conflict': bool,
'trigger': toList(trigger), 'trigger': trigger,
'success': variable_dict, 'success': variable_dict,
'failure': variable_dict, 'failure': variable_dict,
'start': variable_dict, 'start': variable_dict,

View File

@ -47,6 +47,7 @@ class Pipeline(object):
self.manager = None self.manager = None
self.queues = [] self.queues = []
self.precedence = PRECEDENCE_NORMAL self.precedence = PRECEDENCE_NORMAL
self.trigger = None
def __repr__(self): def __repr__(self):
return '<Pipeline %s>' % self.name return '<Pipeline %s>' % self.name
@ -724,6 +725,7 @@ class TriggerEvent(object):
# common # common
self.type = None self.type = None
self.project_name = None self.project_name = None
self.trigger_name = None
# Representation of the user account that performed the event. # Representation of the user account that performed the event.
self.account = None self.account = None
# patchset-created, comment-added, etc. # patchset-created, comment-added, etc.

View File

@ -74,7 +74,7 @@ class Scheduler(threading.Thread):
self._exit = False self._exit = False
self._stopped = False self._stopped = False
self.launcher = None self.launcher = None
self.trigger = None self.triggers = dict()
self.config = None self.config = None
self._maintain_trigger_cache = False self._maintain_trigger_cache = False
@ -141,20 +141,24 @@ class Scheduler(threading.Thread):
manager.success_action = conf_pipeline.get('success') manager.success_action = conf_pipeline.get('success')
manager.failure_action = conf_pipeline.get('failure') manager.failure_action = conf_pipeline.get('failure')
manager.start_action = conf_pipeline.get('start') manager.start_action = conf_pipeline.get('start')
for trigger in toList(conf_pipeline['trigger']): # TODO: move this into triggers (may require pluggable
approvals = {} # configuration)
for approval_dict in toList(trigger.get('approval')): if 'gerrit' in conf_pipeline['trigger']:
for k, v in approval_dict.items(): pipeline.trigger = self.triggers['gerrit']
approvals[k] = v for trigger in toList(conf_pipeline['trigger']['gerrit']):
f = EventFilter(types=toList(trigger['event']), approvals = {}
branches=toList(trigger.get('branch')), for approval_dict in toList(trigger.get('approval')):
refs=toList(trigger.get('ref')), for k, v in approval_dict.items():
approvals=approvals, approvals[k] = v
comment_filters= f = EventFilter(types=toList(trigger['event']),
toList(trigger.get('comment_filter')), branches=toList(trigger.get('branch')),
email_filters= refs=toList(trigger.get('ref')),
toList(trigger.get('email_filter'))) approvals=approvals,
manager.event_filters.append(f) comment_filters=
toList(trigger.get('comment_filter')),
email_filters=
toList(trigger.get('email_filter')))
manager.event_filters.append(f)
for project_template in data.get('project-templates', []): for project_template in data.get('project-templates', []):
# Make sure the template only contains valid pipelines # Make sure the template only contains valid pipelines
@ -272,17 +276,23 @@ class Scheduler(threading.Thread):
else: else:
sshkey = None sshkey = None
self.merger = merger.Merger(self.trigger, merge_root, push_refs, # TODO: The merger should have an upstream repo independent of
# triggers, and then each trigger should provide a fetch
# location.
self.merger = merger.Merger(self.triggers['gerrit'],
merge_root, push_refs,
sshkey, merge_email, merge_name) sshkey, merge_email, merge_name)
for project in self.layout.projects.values(): for project in self.layout.projects.values():
url = self.trigger.getGitUrl(project) url = self.triggers['gerrit'].getGitUrl(project)
self.merger.addProject(project, url) self.merger.addProject(project, url)
def setLauncher(self, launcher): def setLauncher(self, launcher):
self.launcher = launcher self.launcher = launcher
def setTrigger(self, trigger): def registerTrigger(self, trigger, name=None):
self.trigger = trigger if name is None:
name = trigger.name
self.triggers[name] = trigger
def getProject(self, name): def getProject(self, name):
self.layout_lock.acquire() self.layout_lock.acquire()
@ -518,7 +528,8 @@ class Scheduler(threading.Thread):
relevant.add(item.change) relevant.add(item.change)
relevant.update(item.change.getRelatedChanges()) relevant.update(item.change.getRelatedChanges())
self.log.debug("Trigger cache size: %s" % len(relevant)) self.log.debug("Trigger cache size: %s" % len(relevant))
self.trigger.maintainCache(relevant) for trigger in self.triggers.values():
trigger.maintainCache(relevant)
def process_event_queue(self): def process_event_queue(self):
self.log.debug("Fetching trigger event") self.log.debug("Fetching trigger event")
@ -540,7 +551,8 @@ class Scheduler(threading.Thread):
self.merger.updateRepo(project) self.merger.updateRepo(project)
for pipeline in self.layout.pipelines.values(): for pipeline in self.layout.pipelines.values():
change = event.getChange(project, self.trigger) change = event.getChange(project,
self.triggers.get(event.trigger_name))
if event.type == 'patchset-created': if event.type == 'patchset-created':
pipeline.manager.removeOldVersionsOfChange(change) pipeline.manager.removeOldVersionsOfChange(change)
if pipeline.manager.eventMatches(event): if pipeline.manager.eventMatches(event):
@ -709,7 +721,7 @@ class BasePipelineManager(object):
msg = "Starting %s jobs." % self.pipeline.name msg = "Starting %s jobs." % self.pipeline.name
if self.sched.config.has_option('zuul', 'status_url'): if self.sched.config.has_option('zuul', 'status_url'):
msg += "\n" + self.sched.config.get('zuul', 'status_url') msg += "\n" + self.sched.config.get('zuul', 'status_url')
ret = self.sched.trigger.report(change, msg, self.start_action) ret = self.pipeline.trigger.report(change, msg, self.start_action)
if ret: if ret:
self.log.error("Reporting change start %s received: %s" % self.log.error("Reporting change start %s received: %s" %
(change, ret)) (change, ret))
@ -1025,8 +1037,8 @@ class BasePipelineManager(object):
succeeded = self.pipeline.didAllJobsSucceed(item) succeeded = self.pipeline.didAllJobsSucceed(item)
merged = (not ret) merged = (not ret)
if merged: if merged:
merged = self.sched.trigger.isMerged(item.change, merged = self.pipeline.trigger.isMerged(item.change,
item.change.branch) item.change.branch)
self.log.info("Reported change %s status: all-succeeded: %s, " self.log.info("Reported change %s status: all-succeeded: %s, "
"merged: %s" % (item.change, succeeded, merged)) "merged: %s" % (item.change, succeeded, merged))
if not (succeeded and merged): if not (succeeded and merged):
@ -1054,7 +1066,7 @@ class BasePipelineManager(object):
try: try:
self.log.info("Reporting change %s, action: %s" % self.log.info("Reporting change %s, action: %s" %
(item.change, action)) (item.change, action))
ret = self.sched.trigger.report(item.change, report, action) ret = self.pipeline.trigger.report(item.change, report, action)
if ret: if ret:
self.log.error("Reporting change %s received: %s" % self.log.error("Reporting change %s received: %s" %
(item.change, ret)) (item.change, ret))
@ -1307,8 +1319,8 @@ class DependentPipelineManager(BasePipelineManager):
self.log.info(" %s" % queue) self.log.info(" %s" % queue)
def isChangeReadyToBeEnqueued(self, change): def isChangeReadyToBeEnqueued(self, change):
if not self.sched.trigger.canMerge(change, if not self.pipeline.trigger.canMerge(change,
self.getSubmitAllowNeeds()): self.getSubmitAllowNeeds()):
self.log.debug("Change %s can not merge, ignoring" % change) self.log.debug("Change %s can not merge, ignoring" % change)
return False return False
return True return True
@ -1320,8 +1332,8 @@ class DependentPipelineManager(BasePipelineManager):
self.log.debug(" Changeish does not support dependencies") self.log.debug(" Changeish does not support dependencies")
return return
for needs in change.needed_by_changes: for needs in change.needed_by_changes:
if self.sched.trigger.canMerge(needs, if self.pipeline.trigger.canMerge(needs,
self.getSubmitAllowNeeds()): self.getSubmitAllowNeeds()):
self.log.debug(" Change %s needs %s and is ready to merge" % self.log.debug(" Change %s needs %s and is ready to merge" %
(needs, change)) (needs, change))
to_enqueue.append(needs) to_enqueue.append(needs)
@ -1358,8 +1370,8 @@ class DependentPipelineManager(BasePipelineManager):
if self.isChangeAlreadyInQueue(change.needs_change): if self.isChangeAlreadyInQueue(change.needs_change):
self.log.debug(" Needed change is already ahead in the queue") self.log.debug(" Needed change is already ahead in the queue")
return True return True
if self.sched.trigger.canMerge(change.needs_change, if self.pipeline.trigger.canMerge(change.needs_change,
self.getSubmitAllowNeeds()): self.getSubmitAllowNeeds()):
self.log.debug(" Change %s is needed" % self.log.debug(" Change %s is needed" %
change.needs_change) change.needs_change)
return change.needs_change return change.needs_change

View File

@ -25,11 +25,12 @@ class GerritEventConnector(threading.Thread):
log = logging.getLogger("zuul.GerritEventConnector") log = logging.getLogger("zuul.GerritEventConnector")
def __init__(self, gerrit, sched): def __init__(self, gerrit, sched, trigger):
super(GerritEventConnector, self).__init__() super(GerritEventConnector, self).__init__()
self.daemon = True self.daemon = True
self.gerrit = gerrit self.gerrit = gerrit
self.sched = sched self.sched = sched
self.trigger = trigger
self._stopped = False self._stopped = False
def stop(self): def stop(self):
@ -42,6 +43,7 @@ class GerritEventConnector(threading.Thread):
return return
event = TriggerEvent() event = TriggerEvent()
event.type = data.get('type') event.type = data.get('type')
event.trigger_name = self.trigger.name
change = data.get('change') change = data.get('change')
if change: if change:
event.project_name = change.get('project') event.project_name = change.get('project')
@ -85,9 +87,9 @@ class GerritEventConnector(threading.Thread):
# Call getChange for the side effect of updating the # Call getChange for the side effect of updating the
# cache. Note that this modifies Change objects outside # cache. Note that this modifies Change objects outside
# the main thread. # the main thread.
self.sched.trigger.getChange(event.change_number, self.trigger.getChange(event.change_number,
event.patch_number, event.patch_number,
refresh=True) refresh=True)
self.sched.addEvent(event) self.sched.addEvent(event)
self.gerrit.eventDone() self.gerrit.eventDone()
@ -103,6 +105,7 @@ class GerritEventConnector(threading.Thread):
class Gerrit(object): class Gerrit(object):
name = 'gerrit'
log = logging.getLogger("zuul.Gerrit") log = logging.getLogger("zuul.Gerrit")
replication_timeout = 60 replication_timeout = 60
replication_retry_interval = 5 replication_retry_interval = 5
@ -128,7 +131,7 @@ class Gerrit(object):
self.gerrit = gerrit.Gerrit(self.server, user, port, sshkey) self.gerrit = gerrit.Gerrit(self.server, user, port, sshkey)
self.gerrit.startWatching() self.gerrit.startWatching()
self.gerrit_connector = GerritEventConnector( self.gerrit_connector = GerritEventConnector(
self.gerrit, sched) self.gerrit, sched, self)
self.gerrit_connector.start() self.gerrit_connector.start()
def stop(self): def stop(self):