diff --git a/doc/source/zuul.rst b/doc/source/zuul.rst index fc01614979..bcb7a5285f 100644 --- a/doc/source/zuul.rst +++ b/doc/source/zuul.rst @@ -13,54 +13,29 @@ At a Glance :Hosts: * https://zuul.openstack.org + * ze*.openstack.org * zm*.openstack.org :Puppet: * https://git.openstack.org/cgit/openstack-infra/puppet-zuul/tree/ * :cgit_file:`modules/openstack_project/manifests/zuul_prod.pp` * :cgit_file:`modules/openstack_project/manifests/zuul_dev.pp` :Configuration: - * :config:`zuul/layout.yaml` + * :config:`zuul/` + * :config:`zuul.d/` :Projects: - * https://git.openstack.org/cgit/openstack-infra/zuul + * https://git.zuul-ci.org/cgit/zuul :Bugs: - * https://storyboard.openstack.org/#!/project/679 + * https://storyboard.openstack.org/#!/project/openstack-infra/zuul :Resources: - * `Zuul Reference Manual `_ + * `Zuul Reference Manual `_ :Chat: * #zuul on freenode Overview ======== -The OpenStack project uses a number of pipelines in Zuul: - -**check** - Newly uploaded patchsets enter this pipeline to receive an initial - +/-1 Verified vote. - -**gate** - Changes that have been approved by core reviewers are enqueued in - order in this pipeline, and if they pass tests, will be merged. - -**post** - This pipeline runs jobs that operate after each change is merged. - -**pre-release** - This pipeline runs jobs on projects in response to pre-release tags. - -**release** - When a commit is tagged as a release, this pipeline runs jobs that - publish archives and documentation. - -**silent** - This pipeline is used for silently testing new jobs. - -**experimental** - This pipeline is used for on-demand testing of new jobs. - -**periodic** - This pipeline has jobs triggered on a timer for e.g. testing for - environmental changes daily. +The OpenStack project uses a number of pipelines in Zuul, as defined +in :config:`zuul.d/pipelines.yaml`. Zuul watches events in Gerrit (using the Gerrit "stream-events" command) and matches those events to the pipelines above. If a match @@ -77,9 +52,10 @@ each commit is correctly tested. Zuul's current status may be viewed at ``_. -Zuul's configuration is stored in :config:`zuul/layout.yaml`. Anyone -may propose a change to the configuration by editing that file and -submitting the change to Gerrit for review. +Zuul's configuration is distributed across projects listed in +:config:`zuul/main.yaml`. Anyone may propose a change to the +configuration by editing configuration in those projects and submitting +the change to Gerrit for review. For the full syntax of Zuul's configuration file format, see the `Zuul reference manual`_. @@ -89,11 +65,13 @@ Sysadmin Zuul and gear are lightweight - it should be possible to run both on a 1G instance for small deployments. OpenStack's deployment requires at -least a 2G instance at the time of writing. +least a 8G instance at the time of writing, though additional cache +memory helps performance. -Zuul is stateless, so the server does not need backing up. However -zuul talks through git and ssh so you will need to manually check ssh -host keys as the zuul user. e.g.:: +Zuul is mostly stateless, so the server does not need backing up (though +it does rely on a Trove instance for its build history). However zuul +talks through git and ssh so you will need to manually check ssh host +keys as the zuul user. e.g.:: sudo su - zuul ssh -p 29418 review.openstack.org @@ -116,7 +94,7 @@ merged, wait until that has been completed. Since Zuul is stateless, some work needs to be done to save and then re-enqueue patches when restarts are done. To accomplish this, start by running `zuul-changes.py -`_ +`_ to save the check and gate queues:: python /opt/zuul/tools/zuul-changes.py http://zuul.openstack.org \ @@ -127,40 +105,35 @@ to save the check and gate queues:: These check.sh and gate.sh scripts will be used after the restart to re-enqueue the changes. -Now use `service zuul stop` to stop zuul and then run ps to make sure -the process has actually stopped, it may take several seconds for it to -finally go away. +Now use `service zuul-scheduler stop` to stop zuul and then run ps to +make sure the process has actually stopped, it may take several seconds +for it to finally go away. -With Zuul stopped, delete all the used nodes in nodepool. Wait for one -of each variety to come up before using `service zuul start` to start -zuul again. - -Once Zuul is started, run netcat against localhost 4730 port to confirm -that all the node types (particularly the uncommon ones) are registered -with Gearman before re-enqueuing patches. For instance:: - - echo "status" | nc localhost 4730 | grep :centos7 - -When you are satisfied that all the node types have returned, first run -the gate.sh script and then check.sh to re-enqueue the changes from -before the restart:: +When you are satisfied that zuul is up, first run the gate.sh script and +then check.sh to re-enqueue the changes from before the restart:: ./gate.sh ./check.sh You may watch the `Zuul Status Page `_ to confirm that changes are -returning to the queues. +returning to the queues. This frontend is provided by the zuul-web +service on the same server, which may also need to be restarted. + +Executors +--------- + +Servers with names matching the pattern ze*.openstack.org are Zuul +Executors. These are horizontally scalable components of Zuul which +run Ansible within a Bubblewrap context and connect to job nodes. +They can be started and stopped at will, and new ones added as +necessary to accommodate load. Mergers ------- Servers with names matching the pattern zm*.openstack.org are Zuul Mergers. These are horizontally scalable components of Zuul which -perform git operations for the benefit of jobs. They serve git -repositories via Apache over http, and jobs fetch changes to test from -them. They can be started and stopped at will, and new ones added as -necessary to accommodate load. If you remove a merger, be sure to -leave Apache running for several hours until the last job that may -have been launched with instructions to fetch from that merger has -completed. +perform git operations for the benefit of jobs. They can be started +and stopped at will, and new ones added as necessary to accommodate +load.