This switches from the ansible/dhall operator framework to kopf,
an operator framework written in pure Python. This allows us to:
* Build the operator application as a Python app.
* Build the operator image using the opendev python builder images.
* Run the operator as a Python CLI program "zuul-operator".
* Write procedural Python code to handle operator tasks (such as
creating new nodepool launchers when providers are added).
* Use Jinja for templating config files and k8s resource files
(direct pythonic manipulation of resources is an option too).
The new CR nearly matches the existing one, with some minor differences.
Some missing features and documentation are added in the commits
immediately following; they should be reviewed and merged as a unit.
Also, fx waiting for scheduler to settle in functional test since
we changed this log line in Zuul.
Change-Id: Ib37b67e3444b7cd44692d48eee77775ee9049e9f
Change-Id: I70ec31ecd8fe264118215944022b2e7b513dced9
This change fixes an issue where zuul services can't create
a lockfile.
Another issue happening with older operator-framework and newer
kubernetes version is being fixed by updating the operator-framework
following this migration documentation:
https://sdk.operatorframework.io/docs/building-operators/ansible/migration/
Change-Id: I6adfb907184112f0b7debb050975f76cd1dd4b01
It seems like the zuul-operator-promote-image job does not
upload the image. This change adds a new gate job to upload
the image and fix the promote job.
Depends-On: https://review.opendev.org/727868
Change-Id: Ic3b41e3b29182135f71dc1adf8c6ebd9c26d798d
This change updates the dhall code to the latest version.
* Support for record pun where `{ x = x }` can now be written as `{ x }`
* Indentation and function definitions are more compact
This change has been generated after installing dhall v1.33 using this
command: find . -name "*.dhall" -exec dhall --ascii format --inplace {} \;
Change-Id: I3b6560f26e28622aa51150dc8083d127d89a8a7b
This change ensures the pip tool is installed as it is no longer
provided by the OpenDev test instance.
Change-Id: Ie9ebf946fbf847f5574c3e140f43bdb1c9949137
This change simplifies ZooKeeper service configuration to
better describe the situation where the user provides
a zookeeper service.
Change-Id: Ic872b0e4a07acc2dedd79723e6189af2374da58a
This change adds an imagePrefix attribute to enable using
custom images for all the service at once using a single attribute
which default to "docker.io/zuul".
Change-Id: Ia1a5dd4aa9de410937f59a769019dcdb34167944
This change threads the components input to their resources
function so that the container can be configured with the
optional custom image name when set.
Change-Id: I496cc0237f378057b8125f673a760dcda80ecd99
This change adds an optional preview configuration to the spec:
preview:
image: docker.io/zuul/zuul-preview:latest
count: 0
Change-Id: Id9d902b3b9f25b1bf3fa459634396d7520681417
This change adds a Cerficiate resource to manage
the registry tls secret with the cert-manager service.
This change also splits the registry rw user to a
dedicated secret to enable separate creation of the passwords.
Change-Id: I673ea8db31fd2926c82a4288fd9362f225794da8
This change adds a Certificate resource to manage
the gearman tls secret with the cert-manager service.
To keep things simple, this change also merges the client
and server certificates into one secret.
Change-Id: I26e1075ccc5d9ff18bd5d2c68ffdf97244f3f87c
This change adds a new input toggle to enable using a cert-manager service.
The operator currently only setup a selfSigned CA.
Change-Id: Ifc63768a87f9508c66e4414d5286bae2969985e7
This change moves the optional Backend service to dedicated
components file for easier maintainance.
Change-Id: If3f1ce2f473ae916a64ba3a88fd72f2508288f1a
This change refactors the mkDeployment and mkStatefulset function to
enable spliting the component to their own file. The main difference
is that the method needs an extra input for the application name.
Change-Id: I1e7ac6d91f92c0e8db99ec42c76d0a41ede87d84
This change refactors the mkService function for easier maintainance
and usage outside of the resource function.
Change-Id: I33ee419a66a928e953ad2ceea286301082f457aa
This change adds an optional registry configuration to the spec:
registry:
image: docker.io/zuul/zuul-registry:latest
count: 0
storage-size: 20
public-url: https://registry:9000
The operator expect a {{ cr_name }}-registry-tls secret to be provided
for tls and user configuration. If the secret is missing, the operator
creates self signed certificates and generates the user password.
Depends-On: https://review.opendev.org/710644
Change-Id: I0c054485b0ad01d53ddcff93f7bcbf34d1810325
This change replaces the hardcoded internal database password with a
secret environment key provided by the ansible role.
This change also adds the missing DB and ZK environment key required
by the web service.
Change-Id: I4f04732491c627e16988e81bc8ba9cccc78b2da1
This change updates the json_to_dhall task to provides a better
failure message.
This change also refactors the CR spec transformation
using jinja combinaison filter to be able to use the raw spec directly.
Finally this change removes a copy of the spec from the local playbook
by using the flat cr_spec file directly.
Change-Id: I6ed5c6746f01a8a277d50bdef317e56dfde5f66d
In some case, the test node needs more time to pull the
zuul image. This change increases the wait time to 8 minutes
to prevent false positive failure.
Thsi change also:
* adds a build artifact with the generated kubernetes resources.
* redirects post commands output to logfiles to unclutter the
job-output console.
* replaces kubectl wait by rollout status.
* wait for operator and nodepool-launcher deployment.
Change-Id: I1c499bd11576f92b98511cd1ff180026b8aa70d8
This change updates the external_config and job_volumes attribute
name to be consistent with kubernetes definitions.
Depends-On: https://review.opendev.org/707190
Change-Id: I168128e543e03935c03ab9547f208f18e432e0fc
This change implements https://review.opendev.org/706639
to support custom volume to be mounted in job context.
Change-Id: Ic4a6a27707205594910ad90a5577fbbe727e2352
This change update the kubernetes binding to use the new
Optional types. The main consequence is that all the fields that
are optional needs to be prefixed with Some. This let us remove
the `--omit-empty` parameter resulting in cleaner resources where
we don't need to set a dummy emptyDir medium value.
See this issue for the details:
https://github.com/dhall-lang/dhall-kubernetes/issues/86
Change-Id: I23a0a028909208cd58f57a6f07ee93090b3f3a1a
