33 Commits

Author SHA1 Message Date
Clark Boylan
8c088d17f1 Enable nodepool delete after upload option
This enables the nodepool delete-after-upload option with keep-formats
set to qcow2 on x86 image builders. This should clear out vhd and raw
files after uploads for those formats are completed keeping only qcow2
longer term. This should reduce disk space overhead while still enabling
us to convert from qcow2 to the other formats if that becomes necessary.

Note that we do not enable this for arm64 before arm64 builders
currently build raw images only and we still want at least one copy of
the image to be kept even if it is raw (and not qcow2).

Change-Id: I6cf481e0f9a5eaff35b5d961a084ae34a49ea6c6
2024-03-26 15:10:36 -07:00
Clark Boylan
aabaf95b49 Remove centos-7 nodepool image builds
This is the last step in cleaning centos-7 out of nodepool. The previous
change will have cleaned up uploads and now we can stop building the
images entirely.

Change-Id: Ie81d6d516cd6cd42ae9797025a39521ceede7b71
2024-03-13 08:30:16 -07:00
Clark Boylan
c41bc6e5c2 Remove debian-buster image builds from nodepool
This should be landed after the parent chagne has landed and nodepool
has successfully deleted all debian-buster image uploads from our cloud
providers. At this point it should be safe to remove the image builds
entirely.

Change-Id: I7fae65204ca825665c2e168f85d3630686d0cc75
2024-02-23 13:23:22 -08:00
Clark Boylan
feff36e424 Drop debian-buster image uploads from nodepool
Debian buster has been replaced by bullseye and bookworm, both of which
are releases we have images for. It is time to remove the unused debian
buster images as a result.

This change follows the process in nodepool docs for removing a provider
[0] (which isn't quite what we are doing) to properly remove images so
that they can be deleted by nodepool before we remove nodepool's
knowledge of them. The followup change will remove the image builds from
nodepool.

[0] https://zuul-ci.org/docs/nodepool/latest/operation.html#removing-a-provider

Depends-On: https://review.opendev.org/c/opendev/base-jobs/+/910015
Change-Id: I37cb3779944ff9eb1b774ecaf6df3c6929596155
2024-02-23 13:19:49 -08:00
Clark Boylan
211fe14946 Remove opensuse-15 image builds from nodepool
This should be landed after the parent chagne has landed and nodepool
has successfully deleted all opensuse-15 image uploads from our cloud
providers. At this point it should be safe to remove the image builds
entirely.

Change-Id: Icc870ce04b0f0b26df673f85dd6380234979906f
2024-02-22 10:27:37 -08:00
Clark Boylan
4a3c87dbcd Set a six hour nodepool image upload timeout
This was the old timeout then some refactoring happened and we ended up
with the openstacksdk timeout of one hour. Since then Nodepool added the
ability to configure the timeout so we set it back to the original six
hour value.

Change-Id: I29d0fa9d0077bd8e95f68f74143b2d18dc62014b
2023-09-15 12:57:25 -07:00
Clark Boylan
3b9c5d2f07 Remove fedora image builds
This removes the fedora image builds from nodepool. At this point
Nodepool should no longer have any knowledge of fedora.

There is potential for other cleanups for things like dib elements, but
leaving those in place doesn't hurt much.

Change-Id: I3e6984bc060e9d21f7ad851f3a64db8bb555b38a
2023-09-06 09:16:34 -07:00
Dr. Jens Harbott
d0c0ddb977 Reduce frequency of image rebuilds
In order to reduce the load on our builder nodes and reduce the strain
on our providers' image stores, build most images only once per week.

Exceptions are ubuntu-jammy, our most often used distro image, which we
keep rebuilding daily, and some other more frequently used images built
every 2 days.

Change-Id: Ibba7f864b15e478fda59c998843c3b2ace0022d8
2023-09-02 13:18:19 +02:00
Dr. Jens Harbott
4c16313ad2 Build debian bookworm images
Release is done, mirror is in place, ready to go.

Adopt using systemd-timesyncd like we do for recent Ubuntu releases.

Change-Id: I3fbdc151177bf2dba81920a4a2e3966f271b50ad
2023-07-03 06:05:36 +00:00
Ian Wienand
bd53b1c2b4
nodepool: fix new linaro provider name in nb04
I have left the -regionone off this, making its naming inconsistent.
This adds it.

Since this cloud is in its bringup phase, I will put the builder in
emergency, clear out the images for the "linaro" provider and then
apply this by hand, so that we don't have old ZK nodes lying around.
We can then merge this to make it consistent.

Change-Id: I23328cbc53b87e1e81d26cc56f99aaad33b415c0
2023-01-25 09:05:28 +11:00
Ian Wienand
212ad67630
nodepool: drop linaro-us
Drop the linaro-us cloud from nb04 uploads and launcher; it is
replaced by the new linaro cloud.  Region is not running any nodes
since I6ef17bb517078de363286ecad9749cb321b4c92c.

nb03 is still in the inventory, but shutdown and in emergency.  We can
remove the config here and cleanup will follow.

Change-Id: Ia049a2e44d2c4add0346e8262b60cdfb2c976539
2023-01-23 15:45:19 +11:00
Ian Wienand
0cf9319b06
nodepool: empty linaro-us cloud
This should remove nodepool's tracking of the diskimages in this
cloud, in preparation for it's removal.

Change-Id: Icf7b00f88a9de8a91510ee231c47eef207da4ea8
2023-01-23 15:45:00 +11:00
Ian Wienand
af14c7e04b
nb04: use linaro region mirror
This is pointing at the mirror in the old linaro region; point it at
the current one.

Change-Id: Ia30b3d45351c6cc97d51f32bc4820b72df5cfbb6
2023-01-19 10:27:37 +11:00
Ian Wienand
ac0375de48
nb03/04: add linaro as a provider
Follow-on to I8c8439ca2ae8dd868bce9d2b015f6b8428b16a84 to add the new
linaro cloud as a provider.

Change-Id: Iaac5af66324cf780f07dca5886868f7cd540e4c6
2023-01-17 08:10:39 +11:00
Ian Wienand
4f0eeec9aa
Add nb04 config
This is a second ARM64 builder, intended to replace nb03.  Add the
initial config.

Change-Id: Ie6a244306efc793214cce3481bb6b49addc361f7
2023-01-11 08:57:12 +11:00
Clark Boylan
9bcfa57214 Remove old nodepool builder configs
nb03.openstack.org and nb04.opendev.org have been cleaned up. Remove
their host specific configs.

Depends-On: https://review.opendev.org/752740
Change-Id: I310534a45a88546ae5537d3016a76693266212bc
2020-09-18 11:13:36 -07:00
Clark Boylan
b97aec5f9d Pause image builds on nb04.opendev.org
We'd like to delete nb04.opendev.org and go back to running two x86
image builders. In order to do that gracefully we need to pause all
image builds on nb04 so that nb01 and nb02 take up the slack. Then once
nb04's images are no longer in use they can be deleted and the builder
can be removed.

Change-Id: Ibe00252de4f602a6c9f48ad1799d844710ddc6dc
2020-09-10 10:30:16 -07:00
Ian Wienand
4ac03a4a83 Remove special nb04 config file
With nb01/02.opendev.org now being Ansible/container builders, all
builders can use the same config.  This moves the nb04 config back to
nodepool.yaml and unpauses everything except gentoo, which is still
constantly failing.

Change-Id: I55cfedb9ac7fa06ba1725f0ec92f7247237dc74b
2020-05-07 15:54:29 +10:00
Ian Wienand
368a8983e5 Remove tumbleweed-plain images
We took a slightly different approach for tumbleweed and dropped
support for the pip-and-virtualenv element for it (and Fedora 31+)
with I7a6a342461d6001c25e55638ba9b7438c28f2519.  So there's no need
for the -plain image now; switch the main image to using the base pip.

This is currently failing to build due to this.

Change-Id: I3333b475ff9d2f2fb96ad5588c6250cba2593e06
2020-04-28 12:55:37 +10:00
Zuul
3c22957f3a Merge "Start building focal images" 2020-04-25 15:32:33 +00:00
Monty Taylor
f846318458 Start building focal images
Focal is released. Let's start building images for it.

Only build on nb04 - we need newer debootstrap.

Don't built plain images - and don't put pip-and-virtualenv
on these.

Change-Id: I53444cbbd8c46ca4d6919fa7b924134c4d0464a4
2020-04-23 17:37:02 -05:00
James E. Blair
700eeb67ad Revert "Move Ubuntu builds away from nb04"
This reverts commit 03a2a71984f921fdc73466cd7781799d2982369d.

Change I0887a801bb6dd4ce992c39d9e332a18f8194a7b9 merged in nodepool,
which should work around the problem we saw on nb04.  Try again.

Change-Id: I4a13d4f0f07d755a12557b50ec24ba59df51fe97
2020-04-22 12:55:59 -07:00
Ian Wienand
03a2a71984 Move Ubuntu builds away from nb04
Something about the debootstrap processes destroys the nodepool
container, making not only this Ubuntu builds fail but then every
build after.

Move them back to regular builders while we debug further.

Change-Id: I48eb721bffafe2404a332192976848dba2ca80cc
2020-04-20 17:03:55 +10:00
Ian Wienand
054a73ea3e Fix release for Fedora 31
We accidentally set this back to 30 in the reorg

Change-Id: I243eccbc6ee412cf1ff29979dd0fdcc4d577d3c8
2020-04-20 13:25:19 +10:00
Ian Wienand
a3f20a1347 nodepool: Add more plain images
Add more images without pip-and-virtualenv we can test on.  Build all
-plain images on nb04 (the containerised builder) as an additional
test of that environment (the existing bionic-plain build is moved
here too).

Change-Id: Ic29a25527e259a61ca025325b74acbe64a71774c
2020-04-17 08:41:31 +10:00
Ian Wienand
90d6de37ac Add ubuntu-bionic-plain to all regions
We've been testing with this for a while now, so no reason this can't
be deployed more widely.

Change-Id: I84dae0990cd7e90e7d23caf0c3bdfb6cd8abb138
2020-04-17 08:41:31 +10:00
Ian Wienand
b79ccc9824 nodepool: use job inheritance
This uses the job inheritance feature in the dependent change to
update the config file and remove a lot of duplicated content for
diskimages.

The status-quo is kept with this change; nb04 has everything paused
but the Fedora builds, and vice-versa for the main config file.

Depends-On: https://review.opendev.org/713157
Change-Id: I2e1803d77e702f325c3c4b2af0000bffe98c3e0b
2020-04-16 10:55:20 +00:00
James E. Blair
3e2de09c7a Revert "Move suse builds to nb04, drop pip-and-virtualenv"
This reverts commit cc19b638d6c73e1973544128699f3872c589d051.

This has broken keystone jobs.

Change-Id: Id6377d7196f3e57c0729d88aec458dbd4f4ae776
Depends-On: https://review.opendev.org/#/c/720227/
2020-04-15 15:39:49 +00:00
Ian Wienand
4022dc6491 Add ZUUL_USER_SSH_PUBLIC_KEY to opensuse-15 image
This was forgotten when it was moved to nb04, and zuul can't log in.

Change-Id: Ib37d2969a7fbaafd161cce9cc66e11c64af49895
2020-04-15 17:29:33 +10:00
Ian Wienand
cc19b638d6 Move suse builds to nb04, drop pip-and-virtualenv
Move these images to nb04, the container-based builder.  Pause them on
the other builders.  Due to upstream changes the pip-and-virtualenv
element doesn't work on them any more; remove it.  Work is in progress
to fix our base jobs to operate on platforms such as this, but in the
mean time devstack tests should work.

Change-Id: I0b5ccf6f0b441bdb752b5f066b69edda92f4b37a
2020-04-08 12:38:18 +10:00
Ian Wienand
0b6c46a064 Fedora 31 : remove pip-and-virtualenv
As part of [1] we are removing pip-and-virtualenv.  dib 2.35.0 merged
an accidental breakage for Fedora [2].  We might as well use this as
an opportunity to have this as a "bare" test platform; where we leave
configuration of pip/virtualenv up to jobs.  This will be alongside
the ubuntu-bionic-plain images (see
I85438baf5bb31790a56fe5b38327361f0a2398e9).

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html
[2] https://review.opendev.org/716795

Change-Id: I56a2495e9c768d744ac39073c42a5a1754612f70
2020-04-02 11:16:41 +11:00
Ian Wienand
1e8a472073 Add Fedora 31 builds to nb04
Add Fedora 31 to the opendev builder, which is currently the only
container builder that can build this image.

Depends-On: https://review.opendev.org/716104
Change-Id: I811aa3f796e3d1a3a17832bdc038853537f48497
2020-03-31 15:27:22 +11:00
Ian Wienand
616d91caa8 Add config for nb04.opendev.org
Add a config for nb04.opendev.org that has *only* the Fedora image
unpaused.  This is to be the new container-based builder, where we
only want it to build/manage this image, for now.

Change-Id: I7fce3a616ce52dce3bd3639bad37124a8bb1909b
Story: #2007407
Task: #39064
2020-03-18 17:26:10 +11:00