Fixup some details in the zuul doc
Mostly just formatting and punctuation, plus some outdated bits. Signed-off-by: Dr. Jens Harbott <harbott@osism.tech> Change-Id: I641beb5d65f87173d50c74a4e1f0dba48d006231
This commit is contained in:
Dr. Jens Harbott
parent
d5a738ebf1
commit
6961f380d1
|
|
@ -13,9 +13,9 @@ At a Glance
|
|||
|
||||
:Hosts:
|
||||
* https://zuul.opendev.org
|
||||
* zuul*.openstack.org
|
||||
* ze*.openstack.org
|
||||
* zm*.openstack.org
|
||||
* zuul*.opendev.org
|
||||
* ze*.opendev.org
|
||||
* zm*.opendev.org
|
||||
:Configuration:
|
||||
* :config:`zuul/main.yaml`
|
||||
* :config:`zuul.d`
|
||||
|
|
@ -26,12 +26,12 @@ At a Glance
|
|||
:Resources:
|
||||
* `Zuul Reference Manual`_
|
||||
:Chat:
|
||||
* #zuul on OFTC
|
||||
* ``#zuul:opendev.org`` on Matrix
|
||||
|
||||
Overview
|
||||
========
|
||||
|
||||
The OpenStack project uses a number of pipelines in Zuul:
|
||||
The OpenDev project uses a number of pipelines in Zuul:
|
||||
|
||||
**check**
|
||||
Newly uploaded patchsets enter this pipeline to receive an initial
|
||||
|
|
@ -97,7 +97,7 @@ Zuul has three main subsystems:
|
|||
* Zuul Executors
|
||||
* Zuul Web
|
||||
|
||||
that in OpenStack's deployment depend on four 'external' systems:
|
||||
that in OpenDev's deployment depend on four 'external' systems:
|
||||
|
||||
* Nodepool
|
||||
* Zookeeper
|
||||
|
|
@ -111,7 +111,7 @@ The Zuul Scheduler and gear are all co-located on a single host,
|
|||
referred to by the ``zuul.opendev.org`` CNAME in DNS.
|
||||
|
||||
Zuul is stateless, so the server does not need backing up. However
|
||||
zuul talks through git and ssh so you will need to manually check ssh
|
||||
Zuul talks through git and ssh so you will need to manually check ssh
|
||||
host keys as the zuul user.
|
||||
|
||||
e.g.::
|
||||
|
|
@ -122,7 +122,7 @@ e.g.::
|
|||
The Zuul Scheduler talks to Nodepool using Zookeeper and distributes work to
|
||||
the executors using gear.
|
||||
|
||||
OpenStack's Zuul installation is also configured to write job results into
|
||||
OpenDev's Zuul installation is also configured to write job results into
|
||||
a MySQL database via the SQL Reporter plugin. The database for that is a
|
||||
Rackspace Cloud DB and is configured in the ``mysql`` entry of the
|
||||
``zuul_connection_secrets`` entry for the ``zuul-scheduler`` group.
|
||||
|
|
@ -131,12 +131,12 @@ Executors
|
|||
---------
|
||||
|
||||
The Zuul Executors are a horizontally scalable set of servers named
|
||||
ze*.openstack.org. They perform git merging operations for the scheduler
|
||||
``ze*.opendev.org``. They perform git merging operations for the scheduler
|
||||
and execute Ansible playbooks to actually run jobs.
|
||||
|
||||
Our jobs are configured to upload as much information as possible along with
|
||||
their logs, but if there is an error which can not be diagnosed in that
|
||||
manner, logs are available in the executor-debug log file on
|
||||
manner, logs are available in the ``executor-debug`` log file on
|
||||
the executor host. You may use the Zuul build UUID to track
|
||||
assignment of a given job from the Zuul scheduler to the Zuul executor
|
||||
used by that job.
|
||||
|
|
@ -148,8 +148,8 @@ Web
|
|||
---
|
||||
|
||||
Zuul Web is a horizontally scalable service. It is currently running colocated
|
||||
with the scheduler on zuul.openstack.org. Zuul Web provides live console
|
||||
streaming and will be the home of various web dashboards such as the status
|
||||
with the scheduler on ``zuul.opendev.org``. Zuul Web provides live console
|
||||
streaming and is the home of various web dashboards such as the status
|
||||
page.
|
||||
|
||||
Zuul Web is stateless so is safe to restart, however restarting it will result
|
||||
|
|
@ -161,8 +161,8 @@ Restarting Zuul Services
|
|||
|
||||
Currently the safest way to restart the Zuul scheduler is to restart all
|
||||
services at the same time. The reason for this is that if the scheduler is
|
||||
restarted but executors are not then the executors and scheduler can get out
|
||||
of sync with each other. Note that restarting zuul web or a single executor
|
||||
restarted, but executors are not, then the executors and scheduler can get out
|
||||
of sync with each other. Note that restarting Zuul Web or a single executor
|
||||
should continue to be safe as noted above, but this process should generally
|
||||
be preferred.
|
||||
|
||||
|
|
@ -173,29 +173,29 @@ releases or have other important changes that are about to land.
|
|||
|
||||
Since Zuul is stateless, some work needs to be done to save and then
|
||||
re-enqueue patches when restarts are done. To accomplish this, start by
|
||||
running the zuul-changes script to save the check and gate queues::
|
||||
running the ``zuul-changes`` script to save the check and gate queues::
|
||||
|
||||
root@zuul02# ~root/zuul-changes.py https://zuul.opendev.org >queues-$(date +%Y%m%d).sh
|
||||
|
||||
This script will be executed when Zuul is up and running again to restore
|
||||
The resulting script will be executed when Zuul is up and running again to restore
|
||||
the previous queue contents.
|
||||
|
||||
One other thing to consider before restarting all zuul services is you may
|
||||
want to update all of the zuul docker images. This can be useful if restarting
|
||||
Zuul to correct a bug that was fixed in the Zuul codebase. To do this run
|
||||
the zuul_pull.yaml playbook from bridge::
|
||||
the ``zuul_pull.yaml`` playbook from bridge::
|
||||
|
||||
root@bridge# ansible-playbook -f 20 /home/zuul/src/opendev.org/opendev/system-config/playbooks/zuul_pull.yaml
|
||||
root@bridge# ansible-playbook -f20 /home/zuul/src/opendev.org/opendev/system-config/playbooks/zuul_pull.yaml
|
||||
|
||||
Once ready to restart all Zuul services you will want to run the
|
||||
zuul_restart.yaml playbook from bridge to do this::
|
||||
``zuul_restart.yaml`` playbook from bridge to do this::
|
||||
|
||||
root@bridge# ansible-playbook -f20 /home/zuul/src/opendev.org/opendev/system-config/playbooks/zuul_restart.yaml
|
||||
|
||||
Once this playbook is done running the services will have been restarted, but
|
||||
Once this playbook is done running, the services will have been restarted, but
|
||||
the Zuul system still needs to load its configs before it is ready to do work.
|
||||
The `root <https://zuul.opendev.org/>`_ of the Zuul dashboard will show you
|
||||
loaded tenants. Once all tenants show up on this page it is safe to proceed
|
||||
loaded tenants. Once all tenants show up on this page, it is safe to proceed
|
||||
with re-enqueing changes to pipelines with the script we generated earlier.
|
||||
Note that the OpenStack tenant takes the most time. If you wait for it to
|
||||
show up in the dashboard you should be ready to go. You can double check
|
||||
|
|
@ -229,12 +229,12 @@ ZooKeeper. To construct this you will need to know the Zuul connection name
|
|||
and full project name. The connection name in the example above is 'gerrit',
|
||||
replace it with the appropriate connection name for the project you are looking
|
||||
at. Next is the unique project name. In the example above we start with
|
||||
`opendev/system-config` and split it on `/`. Everything before the first `/`
|
||||
is the next component of our name in this case, `opendev`. Then we take the
|
||||
entire name `opendev/system-config` and URL encode it to get
|
||||
`opendev%2Fsystem-config` which becomes our last component.
|
||||
``opendev/system-config`` and split it on ``/``. Everything before the first ``/``
|
||||
is the next component of our name in this case, ``opendev``. Then we take the
|
||||
entire name ``opendev/system-config`` and URL encode it to get
|
||||
``opendev%2Fsystem-config`` which becomes our last component.
|
||||
|
||||
Save the output of this jq command to a file `secret.pem`. Then extract the
|
||||
Save the output of this jq command to a file ``secret.pem``. Then extract the
|
||||
secret ciphertext from the job configuration to remove surrounding
|
||||
YAML (there is no need to recombine split lines) and run the following
|
||||
command to decrypt::
|
||||
|
|
@ -242,8 +242,6 @@ command to decrypt::
|
|||
cat ciphertext.txt | sed 's/^ *//' | base64 -d | sudo openssl rsautl -decrypt -oaep -inkey \
|
||||
secret.pem
|
||||
|
||||
.. _zuul_github_projects:
|
||||
|
||||
Debugging Problems
|
||||
------------------
|
||||
|
||||
|
|
@ -253,13 +251,13 @@ daunting to start as Zuul's logs are quite verbose. The good news is that once
|
|||
you learn a few tricks those verbose logs become quite the powerful tool.
|
||||
|
||||
Often the best place to start is grepping the Zuul scheduler debug log for
|
||||
the pipeline entry identifier (eg change number, tag, or rev sha1)::
|
||||
the pipeline entry identifier (eg change number, tag, or ref sha1)::
|
||||
|
||||
you@zuul02$ grep 123456 /var/log/zuul/debug.log
|
||||
you@zuul02$ grep c6229660cda0af42ecd5afbe7fefdb51136a0436 /var/log/zuul/debug.log
|
||||
|
||||
In many of these log lines you'll see Zuul event IDs like
|
||||
`[e: 1718628fe39643e1bd6a88a9a1477b4f]` this ID identifies the event that
|
||||
``[e: 1718628fe39643e1bd6a88a9a1477b4f]``. This ID identifies the event that
|
||||
triggered Zuul to take action for these changes and is logged through all
|
||||
the Zuul services. It can be very powerful to do a grep on this event ID and
|
||||
trace through the actions that the scheduler took for this event::
|
||||
|
|
@ -272,10 +270,12 @@ ID to grep for actions related to this even on the executor::
|
|||
you@ze01$ grep 1718628fe39643e1bd6a88a9a1477b4f /var/log/zuul/executor-debug.log
|
||||
|
||||
As you trace through the logs related to a change or event ID you can look for
|
||||
`ERROR` or `Traceback` messages to try and identify the underlying source of
|
||||
the problem. Note that `Traceback` messages are not prefixed with the event
|
||||
``ERROR`` or ``Traceback`` messages to try and identify the underlying source of
|
||||
the problem. Note that ``Traceback`` messages are not prefixed with the event
|
||||
ID which means you'll have to grep with additional context, for example using
|
||||
`grep -B20 -A20`.
|
||||
``grep -B20 -A20``.
|
||||
|
||||
.. _zuul_github_projects:
|
||||
|
||||
GitHub Projects
|
||||
===============
|
||||
|
|
@ -291,7 +291,7 @@ navigate to the `OpenDev Zuul`_ app in the GitHub UI. From there they can
|
|||
click "Install", then choose the project or organization they want to install
|
||||
the App on.
|
||||
|
||||
The repository then needs to be added to the `zuul/main.yaml` file before Zuul
|
||||
The repository then needs to be added to the ``zuul/main.yaml`` file before Zuul
|
||||
can be configured to actually run jobs on it.
|
||||
|
||||
Information about the configuration of the OpenDev Zuul App itself can be
|
||||
|
|
@ -299,4 +299,4 @@ found on the :ref:`github` page at :ref:`openstack_zuul_app`.
|
|||
|
||||
.. _OpenDev Zuul: https://github.com/apps/opendev-zuul
|
||||
.. _Zuul Reference Manual: https://zuul-ci.org/docs/zuul
|
||||
.. _Zuul Status Page: http://zuul.opendev.org
|
||||
.. _Zuul Status Page: https://zuul.opendev.org
|
||||
|
|
|
|||
Loading…
Reference in New Issue