20 Commits (63685df30b8d3285573656371d1840cf19a0b377)

Author SHA1 Message Date
James E. Blair 547f6a1b4c Fix zuulRegistryImageVersion
This variable wasn't being used, but should be.

Change-Id: I6621b345cc44d782b5b654406b2747f758450fce
2 years ago
James E. Blair fe827fbeb0 Add support for imagePullSecrets
Since we support custom image prefixes, we should also support
imagePullSecrets since they are likely to be in a private registry.

This also updates some nearby documentation which was out of date.

Change-Id: Id43382284a9adde877e2383644a31bda24030b2b
2 years ago
Albin Vass 5c3eca7d7f Make nodepool external_config mount more generic
This instead mounts every key under externalConfig to /etc/<key>

Change-Id: I6d95a7c71e7da4807cfe5621fc427c6762c58ecf
2 years ago
Albin Vass 40f13b5ca9 Configure debug logs for merger
Change-Id: I3552597d08b9e2fca04132f2a2b0b4d904973346
2 years ago
Albin Vass da43acd1fd Set component command with args instead of command
Otherwise dumb-init is overriden as the entrypoint.

Change-Id: Ied532a3be46a40369f9bea758871cb3e2080bb51
2 years ago
Albin Vass e39d82975e Mount connection sshkeys on executors and mergers
Change-Id: I1628b2d4efe4ed949c70a0a96a5a06c037495a09
2 years ago
James E. Blair 19f80b7d89 Fix error with multiple nodepool providers
An indentation error caused the operator to only create the last
provider listed in the nodepool config.

Additionally, add the nodepool provider to the nodepool launcher
deployment spec.

This is needed to separate the pods for different nodepool launchers
(otherwise, the deployments may consider pods for one launcher to
belong to another).

Change-Id: I76bbae948d1a633028b3565d512fb48d68111dd7
2 years ago
James E. Blair c531a6fb23 Allow terminationGracePeriodSeconds to be configurable
Set a default of 6 hours, but allow users to tell us to wait a
longer or shorter time for executors to gracefully stop.

Change-Id: I12aada7a3c3aaff7b8bbe51744a8dd4d367b6e5a
2 years ago
James E. Blair 3df4bc0ad1 Pass through environment to scheduler, web and launcher
These components may need to have http_proxy set, so let the user
pass env variables.

Change-Id: I6191dd9c8ded70c715f13179c3d904e8f65e144d
2 years ago
James E. Blair d87ea64c06 Add support for zuul-registry
Change-Id: Ie206f8f7c48da3cfaabb33d16f2a339b35db4e5e
2 years ago
James E. Blair 6e0fb7fe1b Support zuul-preview
The test is moved from run.yaml to test.yaml to benefit from the cluster_ip
variable that is set at the start of test.yaml.

Change-Id: Ia0d1cbac94b6b638b6f410d9647ee1822751a767
2 years ago
James E. Blair 245f6d4713 Support fingergw
Change-Id: I5bb8eca947e387f3236c9fb8e0ca6a91772dc338
2 years ago
James E. Blair 5358c6dc0e Support imagePrefix and versions
Allow the user to specify an image prefix and a version tag.

This updates the functional test to account for a recent change
in Zuul which returns incomplete builds on the API endpoint.

Change-Id: I49e82c790a518c1c9f5a463250b5b785eacde55a
2 years ago
James E. Blair ce1f8b338c Add keystore password support
Zuul now requires a keystore password set in zuul.conf.

Change-Id: I2269b2dce156265a084b2a68687a233131933b00
2 years ago
James E. Blair a0b182199e Add merger support
Also, update web and executors to use the replica count from the
spec.

Change-Id: Ifb9ce9d82a18e5b4dd2e30dd7968c2aa32cbeb4f
2 years ago
James E. Blair e503cef9d5 Pass through extra scheduler config options
This lets users specify params like "relative_priority" without
the operator needing to know about them.

Change-Id: I8a82725e51e8c7172063e0fe944a9ee9f13bada1
2 years ago
James E. Blair 088e663782 Support externally managed Zookeeper and DB
The external zookeeper config was mostly a stub and most of the system
actually assumed internal management.  Flesh out the support for external
management.  Do the same for the database.

This also passes through any additional keys in the zookeeper and database
sections of the spec.

Change-Id: I2014616e6366f8d93283e0a6ca38b288aa16a558
2 years ago
James E. Blair b6d8f1eb53 Move ingress to functional test
The operator currently creates an ingress object, but this was really
just to aid testing.  The operator probably shouldn't really create one
since doing so may be site-specific.

Having the ingress greatly aids local and functional testing though, so
move it to the functional test playbook.

Change-Id: Ic6296120f8c107ecc29556fdab4d6cddb5216a50
2 years ago
James E. Blair 1b699279a4 Run a git server in k8s for functional tests
To make the functional tests easier to run locally, run a git server
in k8s instead of on the local machine.

Also, update the secret creation to work with kind as well as minikube.

Change-Id: I4c00129c7105405b0f6256e769486e79c0b173fe
2 years ago
James E. Blair eff9f360f7 Use kopf operator framework
This switches from the ansible/dhall operator framework to kopf,
an operator framework written in pure Python.  This allows us to:

* Build the operator application as a Python app.
* Build the operator image using the opendev python builder images.
* Run the operator as a Python CLI program "zuul-operator".
* Write procedural Python code to handle operator tasks (such as
  creating new nodepool launchers when providers are added).
* Use Jinja for templating config files and k8s resource files
  (direct pythonic manipulation of resources is an option too).

The new CR nearly matches the existing one, with some minor differences.

Some missing features and documentation are added in the commits
immediately following; they should be reviewed and merged as a unit.

Also, fx waiting for scheduler to settle in functional test since
we changed this log line in Zuul.

Change-Id: Ib37b67e3444b7cd44692d48eee77775ee9049e9f

Change-Id: I70ec31ecd8fe264118215944022b2e7b513dced9
2 years ago