Update Zuul service documentation
Much of this document was stale since putting Zuul v3 into production. Attempt to bring it up to our current state. Change-Id: I1b9d665cbf5ca88917d6e0361a6b16026b2ad6c6
This commit is contained in:
Jeremy Stanley
parent
45c427838a
commit
5e668c952d
@ -13,54 +13,29 @@ At a Glance
|
||||
|
||||
:Hosts:
|
||||
* https://zuul.openstack.org
|
||||
* ze*.openstack.org
|
||||
* zm*.openstack.org
|
||||
:Puppet:
|
||||
* https://git.openstack.org/cgit/openstack-infra/puppet-zuul/tree/
|
||||
* :cgit_file:`modules/openstack_project/manifests/zuul_prod.pp`
|
||||
* :cgit_file:`modules/openstack_project/manifests/zuul_dev.pp`
|
||||
:Configuration:
|
||||
* :config:`zuul/layout.yaml`
|
||||
* :config:`zuul/`
|
||||
* :config:`zuul.d/`
|
||||
:Projects:
|
||||
* https://git.openstack.org/cgit/openstack-infra/zuul
|
||||
* https://git.zuul-ci.org/cgit/zuul
|
||||
:Bugs:
|
||||
* https://storyboard.openstack.org/#!/project/679
|
||||
* https://storyboard.openstack.org/#!/project/openstack-infra/zuul
|
||||
:Resources:
|
||||
* `Zuul Reference Manual <http://docs.openstack.org/infra/zuul>`_
|
||||
* `Zuul Reference Manual <https://zuul-ci.org/docs/zuul>`_
|
||||
:Chat:
|
||||
* #zuul on freenode
|
||||
|
||||
Overview
|
||||
========
|
||||
|
||||
The OpenStack project uses a number of pipelines in Zuul:
|
||||
|
||||
**check**
|
||||
Newly uploaded patchsets enter this pipeline to receive an initial
|
||||
+/-1 Verified vote.
|
||||
|
||||
**gate**
|
||||
Changes that have been approved by core reviewers are enqueued in
|
||||
order in this pipeline, and if they pass tests, will be merged.
|
||||
|
||||
**post**
|
||||
This pipeline runs jobs that operate after each change is merged.
|
||||
|
||||
**pre-release**
|
||||
This pipeline runs jobs on projects in response to pre-release tags.
|
||||
|
||||
**release**
|
||||
When a commit is tagged as a release, this pipeline runs jobs that
|
||||
publish archives and documentation.
|
||||
|
||||
**silent**
|
||||
This pipeline is used for silently testing new jobs.
|
||||
|
||||
**experimental**
|
||||
This pipeline is used for on-demand testing of new jobs.
|
||||
|
||||
**periodic**
|
||||
This pipeline has jobs triggered on a timer for e.g. testing for
|
||||
environmental changes daily.
|
||||
The OpenStack project uses a number of pipelines in Zuul, as defined
|
||||
in :config:`zuul.d/pipelines.yaml`.
|
||||
|
||||
Zuul watches events in Gerrit (using the Gerrit "stream-events"
|
||||
command) and matches those events to the pipelines above. If a match
|
||||
@ -77,9 +52,10 @@ each commit is correctly tested.
|
||||
Zuul's current status may be viewed at
|
||||
`<https://zuul.openstack.org/>`_.
|
||||
|
||||
Zuul's configuration is stored in :config:`zuul/layout.yaml`. Anyone
|
||||
may propose a change to the configuration by editing that file and
|
||||
submitting the change to Gerrit for review.
|
||||
Zuul's configuration is distributed across projects listed in
|
||||
:config:`zuul/main.yaml`. Anyone may propose a change to the
|
||||
configuration by editing configuration in those projects and submitting
|
||||
the change to Gerrit for review.
|
||||
|
||||
For the full syntax of Zuul's configuration file format, see the `Zuul
|
||||
reference manual`_.
|
||||
@ -89,11 +65,13 @@ Sysadmin
|
||||
|
||||
Zuul and gear are lightweight - it should be possible to run both on a
|
||||
1G instance for small deployments. OpenStack's deployment requires at
|
||||
least a 2G instance at the time of writing.
|
||||
least a 8G instance at the time of writing, though additional cache
|
||||
memory helps performance.
|
||||
|
||||
Zuul is stateless, so the server does not need backing up. However
|
||||
zuul talks through git and ssh so you will need to manually check ssh
|
||||
host keys as the zuul user. e.g.::
|
||||
Zuul is mostly stateless, so the server does not need backing up (though
|
||||
it does rely on a Trove instance for its build history). However zuul
|
||||
talks through git and ssh so you will need to manually check ssh host
|
||||
keys as the zuul user. e.g.::
|
||||
|
||||
sudo su - zuul
|
||||
ssh -p 29418 review.openstack.org
|
||||
@ -116,7 +94,7 @@ merged, wait until that has been completed.
|
||||
Since Zuul is stateless, some work needs to be done to save and then
|
||||
re-enqueue patches when restarts are done. To accomplish this, start by
|
||||
running `zuul-changes.py
|
||||
<https://git.openstack.org/cgit/openstack-infra/zuul/tree/tools/zuul-changes.py>`_
|
||||
<https://git.zuul-ci.org/cgit/zuul/tree/tools/zuul-changes.py>`_
|
||||
to save the check and gate queues::
|
||||
|
||||
python /opt/zuul/tools/zuul-changes.py http://zuul.openstack.org \
|
||||
@ -127,40 +105,35 @@ to save the check and gate queues::
|
||||
These check.sh and gate.sh scripts will be used after the restart to
|
||||
re-enqueue the changes.
|
||||
|
||||
Now use `service zuul stop` to stop zuul and then run ps to make sure
|
||||
the process has actually stopped, it may take several seconds for it to
|
||||
finally go away.
|
||||
Now use `service zuul-scheduler stop` to stop zuul and then run ps to
|
||||
make sure the process has actually stopped, it may take several seconds
|
||||
for it to finally go away.
|
||||
|
||||
With Zuul stopped, delete all the used nodes in nodepool. Wait for one
|
||||
of each variety to come up before using `service zuul start` to start
|
||||
zuul again.
|
||||
|
||||
Once Zuul is started, run netcat against localhost 4730 port to confirm
|
||||
that all the node types (particularly the uncommon ones) are registered
|
||||
with Gearman before re-enqueuing patches. For instance::
|
||||
|
||||
echo "status" | nc localhost 4730 | grep :centos7
|
||||
|
||||
When you are satisfied that all the node types have returned, first run
|
||||
the gate.sh script and then check.sh to re-enqueue the changes from
|
||||
before the restart::
|
||||
When you are satisfied that zuul is up, first run the gate.sh script and
|
||||
then check.sh to re-enqueue the changes from before the restart::
|
||||
|
||||
./gate.sh
|
||||
./check.sh
|
||||
|
||||
You may watch the `Zuul Status Page
|
||||
<https://zuul.openstack.org/>`_ to confirm that changes are
|
||||
returning to the queues.
|
||||
returning to the queues. This frontend is provided by the zuul-web
|
||||
service on the same server, which may also need to be restarted.
|
||||
|
||||
Executors
|
||||
---------
|
||||
|
||||
Servers with names matching the pattern ze*.openstack.org are Zuul
|
||||
Executors. These are horizontally scalable components of Zuul which
|
||||
run Ansible within a Bubblewrap context and connect to job nodes.
|
||||
They can be started and stopped at will, and new ones added as
|
||||
necessary to accommodate load.
|
||||
|
||||
Mergers
|
||||
-------
|
||||
|
||||
Servers with names matching the pattern zm*.openstack.org are Zuul
|
||||
Mergers. These are horizontally scalable components of Zuul which
|
||||
perform git operations for the benefit of jobs. They serve git
|
||||
repositories via Apache over http, and jobs fetch changes to test from
|
||||
them. They can be started and stopped at will, and new ones added as
|
||||
necessary to accommodate load. If you remove a merger, be sure to
|
||||
leave Apache running for several hours until the last job that may
|
||||
have been launched with instructions to fetch from that merger has
|
||||
completed.
|
||||
perform git operations for the benefit of jobs. They can be started
|
||||
and stopped at will, and new ones added as necessary to accommodate
|
||||
load.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user