Since we support custom image prefixes, we should also support
imagePullSecrets since they are likely to be in a private registry.
This also updates some nearby documentation which was out of date.
We are supposed to update zuul/nodepool when the relevant secrets
are updated. We do that by subscribing to changes on all secrets
in the system, then check to see if they are ones that we're watching
and if so, run the appropriate operator methods.
We only updated the list of secrets to watch on startup, which means
that from a clean start where a zuul was added, we would not add
the new secrets to our list. This change updates the list after every
time that the zuul resource is created or updated.
This switches from the ansible/dhall operator framework to kopf,
an operator framework written in pure Python. This allows us to:
* Build the operator application as a Python app.
* Build the operator image using the opendev python builder images.
* Run the operator as a Python CLI program "zuul-operator".
* Write procedural Python code to handle operator tasks (such as
creating new nodepool launchers when providers are added).
* Use Jinja for templating config files and k8s resource files
(direct pythonic manipulation of resources is an option too).
The new CR nearly matches the existing one, with some minor differences.
Some missing features and documentation are added in the commits
immediately following; they should be reviewed and merged as a unit.
Also, fx waiting for scheduler to settle in functional test since
we changed this log line in Zuul.