Commit Graph

145 Commits (346e3c19f7c3860f6e8c97d8d4d1561206ef5b24)

Author SHA1 Message Date
Zuul 346e3c19f7 Merge "Document externalConfig" 2021-08-18 21:58:52 +00:00
Zuul b62faddca7 Merge "Fix wait for scheduler to settle in tests" 2021-08-17 23:36:11 +00:00
Zuul 8ad92f242e Merge "Make nodepool external_config mount more generic" 2021-08-17 11:53:35 +00:00
James E. Blair 3ee16a7b2f Fix wait for scheduler to settle in tests
The log line that we should wait for has changed in Zuul; this is
the new equivalent.

Change-Id: I34fc52729fa64bb36c72fe6661f0794620a25092
Depends-On: https://review.opendev.org/c/zuul/zuul/+/804806
2021-08-17 00:10:42 +00:00
Zuul b977aa9e12 Merge "Configure debug logs for merger" 2021-08-05 17:42:54 +00:00
Zuul 954b4060a2 Merge "Set component command with args instead of command" 2021-08-05 17:42:53 +00:00
Zuul 345cda5b66 Merge "Mount connection sshkeys on executors and mergers" 2021-08-05 17:42:52 +00:00
Zuul d79e4235dc Merge "Add static node to functional test" 2021-08-05 17:42:51 +00:00
James E. Blair 693cb8e5d8 Document externalConfig
Change-Id: I14c80386553570e4449e82170b0d924085742ee9
2021-08-05 07:35:08 -07:00
Albin Vass 5c3eca7d7f Make nodepool external_config mount more generic
This instead mounts every key under externalConfig to /etc/<key>

Change-Id: I6d95a7c71e7da4807cfe5621fc427c6762c58ecf
2021-08-05 07:35:07 -07:00
Albin Vass 40f13b5ca9 Configure debug logs for merger
Change-Id: I3552597d08b9e2fca04132f2a2b0b4d904973346
2021-08-05 07:35:06 -07:00
Albin Vass da43acd1fd Set component command with args instead of command
Otherwise dumb-init is overriden as the entrypoint.

Change-Id: Ied532a3be46a40369f9bea758871cb3e2080bb51
2021-08-05 07:35:05 -07:00
Albin Vass e39d82975e Mount connection sshkeys on executors and mergers
Change-Id: I1628b2d4efe4ed949c70a0a96a5a06c037495a09
2021-08-05 07:35:04 -07:00
James E. Blair 97aa27080f Add static node to functional test
So that we can better test functions related to ssh connectivity,
add a static node to the functional test.

Change-Id: I3dfdaca0b4e5dbfb35e4a645c7bcb56e0eec88c2
2021-08-05 07:34:49 -07:00
Zuul d5c046d107 Merge "Add instructions and tools for running tests with kind" 2021-07-30 00:06:32 +00:00
Zuul 4648a32e01 Merge "Fix error with multiple nodepool providers" 2021-07-30 00:06:31 +00:00
Zuul 527031255a Merge "Flake8 cleanups" 2021-07-30 00:06:30 +00:00
Zuul e68faab480 Merge "Allow terminationGracePeriodSeconds to be configurable" 2021-07-30 00:06:29 +00:00
Zuul 4107016ff8 Merge "Pass through environment to scheduler, web and launcher" 2021-07-30 00:05:42 +00:00
Zuul 51a44bfc48 Merge "Add allowUnsafeConfig database setting" 2021-07-30 00:05:41 +00:00
Zuul 4277568390 Merge "Remove extra 2 minute wait from tests" 2021-07-30 00:05:40 +00:00
Zuul afdfa5072e Merge "Add support for zuul-registry" 2021-07-30 00:02:29 +00:00
Zuul 763d58df6d Merge "Support zuul-preview" 2021-07-30 00:02:28 +00:00
Zuul c6e50da9e5 Merge "Switch jobs to use fedora-34 nodes" 2021-07-28 06:03:53 +00:00
James E. Blair 901ec92b9d Add instructions and tools for running tests with kind
This is the test setup I have used to run the functional tests
locally.  In particular, the restart.sh script is useful for
very quickly rebuilding the cluster from scratch while also
using the local docker image cache to populate the cluster with
cached images for speed.

Change-Id: I994de1cdd49b487d68083f6e01ba73bfc4274d05
2021-07-23 10:27:11 -07:00
James E. Blair 19f80b7d89 Fix error with multiple nodepool providers
An indentation error caused the operator to only create the last
provider listed in the nodepool config.

Additionally, add the nodepool provider to the nodepool launcher
deployment spec.

This is needed to separate the pods for different nodepool launchers
(otherwise, the deployments may consider pods for one launcher to
belong to another).

Change-Id: I76bbae948d1a633028b3565d512fb48d68111dd7
2021-07-23 10:27:10 -07:00
James E. Blair 9f2f4ec145 Flake8 cleanups
Tox can now run the linters test.

The linters gate job is added.

Change-Id: If39545c928991baa47c919ab6b357bf47ca9ca90
2021-07-23 10:27:10 -07:00
James E. Blair c531a6fb23 Allow terminationGracePeriodSeconds to be configurable
Set a default of 6 hours, but allow users to tell us to wait a
longer or shorter time for executors to gracefully stop.

Change-Id: I12aada7a3c3aaff7b8bbe51744a8dd4d367b6e5a
2021-07-23 10:27:09 -07:00
James E. Blair 3df4bc0ad1 Pass through environment to scheduler, web and launcher
These components may need to have http_proxy set, so let the user
pass env variables.

Change-Id: I6191dd9c8ded70c715f13179c3d904e8f65e144d
2021-07-23 10:27:08 -07:00
James E. Blair 182a563a9e Add allowUnsafeConfig database setting
We overrode the PXC resource values in order for functional tests
in minikube to work.  Restore the defaults and set an override in
the functional tests.

Change-Id: Id0560d9089f03ddcb116dccec0b424b811186d65
2021-07-23 10:27:07 -07:00
James E. Blair e448c52b79 Remove extra 2 minute wait from tests
This doesn't seem to be necessary any more.

Change-Id: I02d4c417bdf9574cd61577044b80fdc8bdd3a779
2021-07-23 10:27:06 -07:00
James E. Blair d87ea64c06 Add support for zuul-registry
Change-Id: Ie206f8f7c48da3cfaabb33d16f2a339b35db4e5e
2021-07-23 10:27:06 -07:00
James E. Blair 6e0fb7fe1b Support zuul-preview
The test is moved from run.yaml to test.yaml to benefit from the cluster_ip
variable that is set at the start of test.yaml.

Change-Id: Ia0d1cbac94b6b638b6f410d9647ee1822751a767
2021-07-23 10:27:05 -07:00
James E. Blair a32c31bc3c Add docs
Change-Id: I2975607a51860df88863454785ae5d189f6f2831
2021-07-23 10:27:04 -07:00
James E. Blair 245f6d4713 Support fingergw
Change-Id: I5bb8eca947e387f3236c9fb8e0ca6a91772dc338
2021-07-23 10:27:02 -07:00
James E. Blair 5358c6dc0e Support imagePrefix and versions
Allow the user to specify an image prefix and a version tag.

This updates the functional test to account for a recent change
in Zuul which returns incomplete builds on the API endpoint.

Change-Id: I49e82c790a518c1c9f5a463250b5b785eacde55a
2021-07-23 09:47:08 -07:00
James E. Blair ce1f8b338c Add keystore password support
Zuul now requires a keystore password set in zuul.conf.

Change-Id: I2269b2dce156265a084b2a68687a233131933b00
2021-07-20 13:20:45 -07:00
James E. Blair a0b182199e Add merger support
Also, update web and executors to use the replica count from the
spec.

Change-Id: Ifb9ce9d82a18e5b4dd2e30dd7968c2aa32cbeb4f
2021-07-20 13:20:40 -07:00
James E. Blair e503cef9d5 Pass through extra scheduler config options
This lets users specify params like "relative_priority" without
the operator needing to know about them.

Change-Id: I8a82725e51e8c7172063e0fe944a9ee9f13bada1
2021-07-20 13:20:37 -07:00
James E. Blair 088e663782 Support externally managed Zookeeper and DB
The external zookeeper config was mostly a stub and most of the system
actually assumed internal management.  Flesh out the support for external
management.  Do the same for the database.

This also passes through any additional keys in the zookeeper and database
sections of the spec.

Change-Id: I2014616e6366f8d93283e0a6ca38b288aa16a558
2021-07-20 13:20:33 -07:00
James E. Blair b6d8f1eb53 Move ingress to functional test
The operator currently creates an ingress object, but this was really
just to aid testing.  The operator probably shouldn't really create one
since doing so may be site-specific.

Having the ingress greatly aids local and functional testing though, so
move it to the functional test playbook.

Change-Id: Ic6296120f8c107ecc29556fdab4d6cddb5216a50
2021-07-20 13:20:07 -07:00
James E. Blair 1b699279a4 Run a git server in k8s for functional tests
To make the functional tests easier to run locally, run a git server
in k8s instead of on the local machine.

Also, update the secret creation to work with kind as well as minikube.

Change-Id: I4c00129c7105405b0f6256e769486e79c0b173fe
2021-07-20 13:19:59 -07:00
James E. Blair e5879666bd Bump API version to v1alpha2
To accomodate the CR syntax changes.

Change-Id: Ie646a9c58655b0987a8fb0d4c914cd8b93190b1b
2021-07-20 13:16:58 -07:00
James E. Blair eff9f360f7 Use kopf operator framework
This switches from the ansible/dhall operator framework to kopf,
an operator framework written in pure Python.  This allows us to:

* Build the operator application as a Python app.
* Build the operator image using the opendev python builder images.
* Run the operator as a Python CLI program "zuul-operator".
* Write procedural Python code to handle operator tasks (such as
  creating new nodepool launchers when providers are added).
* Use Jinja for templating config files and k8s resource files
  (direct pythonic manipulation of resources is an option too).

The new CR nearly matches the existing one, with some minor differences.

Some missing features and documentation are added in the commits
immediately following; they should be reviewed and merged as a unit.

Also, fx waiting for scheduler to settle in functional test since
we changed this log line in Zuul.

Change-Id: Ib37b67e3444b7cd44692d48eee77775ee9049e9f

Change-Id: I70ec31ecd8fe264118215944022b2e7b513dced9
2021-07-20 13:16:07 -07:00
Mohammed Naser 2b7d8cf7d1 Switch jobs to use fedora-34 nodes
Fedora 32 is now EOL, we should test against the newly released
version of Fedora which is 34.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/795604
Change-Id: I40a9d7377a4bf5c7ef88109f9a12d35641820ca7
2021-06-09 14:16:36 -04:00
James E. Blair 0366b867bf Upload release images on tag
This uploads release images to dockerhub when we tag the repo.

Change-Id: Ic44a9d29a1e888d8d7874205b0025f286912b5e8
2021-04-10 18:37:28 -07:00
Tristan Cacqueray ee8a7b2cfb Remove command args override and update operator-framework to v1.4.2
This change fixes an issue where zuul services can't create
a lockfile.

Another issue happening with older operator-framework and newer
kubernetes version is being fixed by updating the operator-framework
following this migration documentation:
  https://sdk.operatorframework.io/docs/building-operators/ansible/migration/

Change-Id: I6adfb907184112f0b7debb050975f76cd1dd4b01
2021-04-06 17:23:56 +00:00
Daniel Pawlik 2c367363e8 Replace Fedora 31 with Fedora 32 in zuul-operator-functional
Fedora 31 is EOL and it should be replaced to new version.

Change-Id: Ib51035850528666ece39a1c3f689d73bce29b802
2020-12-17 09:37:52 +01:00
Tristan Cacqueray 3454835c63 Add zuul-operator-upload-image job
It seems like the zuul-operator-promote-image job does not
upload the image. This change adds a new gate job to upload
the image and fix the promote job.

Depends-On: https://review.opendev.org/727868
Change-Id: Ic3b41e3b29182135f71dc1adf8c6ebd9c26d798d
2020-12-04 16:38:32 +00:00
Tristan Cacqueray 76619d839a Add reformat changes to the blame ignore list
This change prevents git blame from refering the previous
reformatting changes.

Change-Id: I7bcd18e23f0f74b0a09192ee70587703d1fe7eb8
2020-07-08 13:20:57 -04:00