1191 Commits

Author SHA1 Message Date
David Moreau Simard
2a0d9faadb
Bump max-servers limestone nodepool provider to 10
We've manually and successfully tested a virtual machine with the
nodepool image and the ipv6 network.
The virtual machine could reach the regional mirror and a devstack-gate
reproduce.sh ran to completion.

Let's do an initial bump to 10 nodes to monitor for failures we could
not catch -- we'll bump to 50 afterwards if we're confident things are
working well for both us and Limestone.

Change-Id: I3a3b68dd9fbd0243f3544ceceb4859aa6d435fc8
2018-03-26 11:06:47 -04:00
Paul Belanger
82ec89b728
Set launch-timeout to 600 for OVH
When OVH is working as execpted, it usually takes 45 seconds to bring
a node online. When things are not working, it takes much longer.
Rather then waiting the default 3600 seconds to fail, drop to 600 to
help speed up nodes being rescheduled.

Change-Id: I122e1782b95f580517c34019ff16b39a053a8115
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-24 07:36:52 -04:00
David Moreau Simard
db2fbcd009
Add limestone to nodepool builder and launcher configuration
This makes limestone available for uploading images to and for testing.
max-server will be bumped once tests have been conclusive.

Change-Id: If630dca09b57f8ef842f2c01459e00bbf2d5c5dc
Depends-On: https://review.openstack.org/#/c/555470/
2018-03-23 09:25:02 -04:00
Ian Wienand
1018a69b4d Unpause all builds for dib 2.12.1
This is the final revert of 8962dfb8d8e463a1b4d42dfd8f6b2daa116c5b46

Change-Id: Id6482b22a3a1699ecfccb2e1bf8731f13a7b12c3
2018-03-22 12:18:37 +11:00
Ian Wienand
6e1fb811ab Unpause Centos 7 for dib 2.12.1
This is a partial revert of 8962dfb8d8e463a1b4d42dfd8f6b2daa116c5b46
to just unpause CentOS 7 and validate the fix.

Change-Id: I234d428d9eddbeb95b8db681c49e75caa34bb128
2018-03-22 12:16:46 +11:00
Ian Wienand
8962dfb8d8 Pause builds for dib 2.12.0
dib 2.12.0 is creating an invalid main partition.  Pause until fixed.

Change-Id: I920e5230a96bf3e6c4a29486c662e905436ab3a7
2018-03-21 14:33:17 +11:00
Paul Belanger
63290ec3e6
Bring vexxhost back online for testing
We created a new mirror in vexxhost with no bandwidth limits, the
thought here is this should results in better job run times as our
mirror will not be the bottleneck.

We've also changed flavors to use v2-highcpu-8 which are 8C/8Gb
memory, but require us to now boot-from-volume with 80GB volume-size.

Change-Id: Ibb26597cfd3c3b10891d7a60e70c6143d1c12407
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-19 15:00:32 -04:00
Paul Belanger
c919e1e782 Remove jenkins-slave element from DIB images
There is no jenkins, only zuul! With zuulv3 running in production,
there is no more need to add jenkins user or scripts to our images.

Move chmod of /opt/cache/files from
nodepool/elements/jenkins-slave/install.d/20-jenkins-slave to
nodepool/elements/zuul-worker/install.d/60-zuul-worker

We also leave the jenkins-slave elements for now, in case 3rd party CI
use them.

Change-Id: Ia9750877fbc1a17ec467ca4ac685afdb9c1627f8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-05 14:29:26 +01:00
Andreas Jaeger
91400af8ba Good bye jenkins/scripts
We now only have afs-slug.sh in jenkins/scripts/ and it is only
used by nodepool/scripts/configure_mirror.sh. Move the file into the
nodepool/scripts directory and adjust paths.

Remove copying of jenkins/scripts from nodepool, the directory does not
exist anymore.

Remove 89-jenkins-scripts, we don't need it anymore.

Depends-On: https://review.openstack.org/548393
Change-Id: I13cac791d70a2e5bcb0de1ba0adcd79e1b0aee9f
2018-03-05 14:28:42 +01:00
Zuul
a6f195df63 Merge "Remove unused nodepool ready-scripts" 2018-03-05 13:21:52 +00:00
Zuul
20244e93e4 Merge "Remove unnecessary distribution determination" 2018-03-05 09:04:43 +00:00
Zuul
a478173278 Merge "Update zanata-cli to 4.3.3" 2018-03-02 08:27:05 +00:00
Dirk Mueller
6864367ad8 Remove unnecessary distribution determination
The need for the distribution was removed in I249f21a98fea3b963b7ffb8e3d0fce02cc540d46
so we can remove the code around that was determining it but not
actually making use of it anywhere anymore.

Change-Id: I572943bbc0c2665a885b7a80cdb2d1817c61bc3c
2018-03-02 01:55:11 +01:00
Dirk Mueller
b0b73ea971 Stop using connection tracking for ssh connections
There is an issue with newer kernels (it seems to happen with 4.15.x)
that when conntrack is reloaded while a connection has packets in
flight, this connection going forward is neither considered INVALID
nor RELATED nor ESTABLISHED by the stateful tracking. While this is
certainly a bug somewhere in the kernel, we can be easily avoiding
this by just not using stateful filtering for ssh connections, as
we accept any connection from anywhere anyway.

Change-Id: I1b20644ce888930cd28d6eaf2c23787315e8199c
2018-03-01 20:09:50 +01:00
Andreas Jaeger
939cb6d41d Update zanata-cli to 4.3.3
The server is now running version 4.3.3, update the client as well.

Change-Id: I68727dc5fe9efef392bbb8250260b2c4524c1153
2018-03-01 09:03:48 +01:00
Paul Belanger
70eb9ea584
Remove unused nodepool ready-scripts
We no longer have the ability to run ready-scripts in nodepool, so
let's delete / clean them up.

Change-Id: I2551037dcfba2362d23401536295eb8f8f5d3e0d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-27 06:59:54 -05:00
Paul Belanger
8d86f7184f
Add ubuntu-bionic to nodepool
As ubuntu-bionic is the new LTS for ubuntu, start providing images in
nodepool.  Something to note, bionic isn't actually released yet.
However our tooling does work and is able to create workable images.

We should hold off to migrating jobs from ubuntu-xenial to
ubuntu-bionic until it is officially released, and we have a migration
plan in place.

Change-Id: If12435969ca37400bfd04abe8470048146be7aa3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-27 06:56:41 -05:00
Paul Belanger
9f5b8baa96 Remove max-ready-age from nodepool
We've landed a fix in nodepool to help reduce the chance of a provider
wedge. Remove max-ready-age, to allow our nodepool fix to handle the
issue.

Change-Id: I02b7112d4fdfb0315ad964924692b0968bdf76b4
Depends-On: https://review.openstack.org/539316
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-27 09:19:45 +00:00
Zuul
607cacbc54 Merge "Add arm64 min-ready to nl03" 2018-02-23 05:31:27 +00:00
Ian Wienand
653c54d96e Add arm64 min-ready to nl03
The launcher calls self.labelImageIsAvailable when calculating the
min-ready values.  This doesn't work if the launcher doesn't have
entries for that label.

Move the arm64 min-ready into nl03 (and out of nl01) and add a note on
this behaviour.

[1] https://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/launcher.py#n946

Change-Id: I060af7501a8e1106cea673a99fce332fcda73290
2018-02-23 16:14:22 +11:00
Zuul
ff97f421fe Merge "Fix ubuntu-xenial-arm64 label" 2018-02-23 05:02:14 +00:00
Ian Wienand
3d29ca65f6 Fix ubuntu-xenial-arm64 label
I believe I mistyped this in Id97a09a459a2336cf528aa551d022217b85b5f01
and forgot the -arm64

Change-Id: I3b20dd94ac5d892c8d305ad8bd0bf49d54401ae0
2018-02-23 15:51:02 +11:00
Zuul
26288ef510 Merge "Add arm64 nodes to launcher" 2018-02-23 04:05:41 +00:00
Ian Wienand
2b78cddaad Add arm64 nodes to launcher
Add configuration for arm64 nodes to nl03

Change-Id: Id97a09a459a2336cf528aa551d022217b85b5f01
Depends-On: https://review.openstack.org/546834
2018-02-23 10:43:11 +11:00
Zuul
05ace22976 Merge "Move jenkins/data/bindep-fallback.txt (2/2)" 2018-02-22 21:14:31 +00:00
Paul Belanger
a9b3c04b47
Bring online opensuse-tumbleweed images
Now that we've been able to build / upload an images, add
configuration and have nodepool boot it.

Change-Id: Ieaaaa8d5f3c65e41964a0eda18072d298cfb15a8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-21 19:30:07 -05:00
Paul Belanger
2d8f0f5a9a
Reduce duplicate data in nodepool configuration
We can use yaml anchors to help reduce duplicate data in nodepool.yaml
files.

Change-Id: I59b43d87340fc0adf37d251c01a0c3092deb3be7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-21 19:24:08 -05:00
Dirk Mueller
0e8ca3a219 Add building of openSUSE Tumbleweed nodepool images
This is useful for experimental devstack runs that allow testing
e.g. Nova or Neutron against the newest technology as it gets
into the rolling release of openSUSE Tumbleweed (like for
example testing latest OVS, kernel and other OS dependencies).

Depends-On: https://review.openstack.org/545314/
Change-Id: Ied661ad453e6345956d466f15f728ce019affe3e
2018-02-22 00:07:51 +01:00
Zuul
ad6ec8c575 Merge "Move jenkins/data/bindep-fallback.txt (1/2)" 2018-02-21 16:27:08 +00:00
Paul Belanger
526efca653
Remove DIB images from citycloud
Because citycloud is currently disable, remove images over pausing
them. This means we can clean up old DIB images left on disk for our
nodepool-builders.

Change-Id: Ide55ae65c1e152bd72dd6259bd8e7e1d308046f5
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-20 16:40:32 -05:00
Andreas Jaeger
2c8541a21c Move jenkins/data/bindep-fallback.txt (2/2)
Remove using old location, we can switch now to the new location
everywhere.

Note: We can only merge this after new images are build with the new
location build in.

This needs a recheck once change 1 is in since project-config is
not self-testing. So, we can remove jenkins/data/bindep-fallback.txt in
this change.

Change-Id: I08f23e96defb152946ade1490628f37f0215300d
Depends-On: https://review.openstack.org/543141
Depends-On: https://review.openstack.org/543142
2018-02-20 20:52:47 +01:00
Andreas Jaeger
9863abbc72 Move jenkins/data/bindep-fallback.txt (1/2)
Let's get rid of jenkins in the name and use a new place to place
bindep-fallback.txt in.

Create new nodepool element to copy the file over.

This is a first of two changes for this repo, we also need to update
the few users in bindep and openstack-zuul-jobs before we can remove the
old location.

We can remove jenkins/data/bindep-fallback.txt only in the followup
change since project-config is not self-testing.

Change-Id: I5c94dcbbca25db7dd998bfbccfbb25c1c4342a2b
2018-02-20 20:52:17 +01:00
Ian Wienand
32c513a6ee Add nb03.openstack.org config
This is a specific config for a builder to run in the linaro cloud
building ARM64 based images.  It only has the configuration for that
cloud.

Launchers will be updated once we can build an image...

Change-Id: If2d111be6b9fb2fdebde70edbb4645ce401d77c3
Depends-On: https://review.openstack.org/546025
2018-02-20 16:29:33 +11:00
Paul Belanger
50754ca31d
Have nl01.o.o manage min-ready of labels
Due to how multiple nodepool-launchers work today, it is possible to
have greater then min-ready of a label. To avoid this, nl01.o.o will
now be the only launcher to send min-ready requests to zookeeper.

Ideally, nodepool will be updated to support shared configuration in
the future.

Change-Id: I1cf0253343a80146f538ca5e4c83bde5c0173be3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-13 15:04:31 -05:00
Zuul
93a0c17cca Merge "Remove zuul-sudo-grep lines from nodepool" 2018-02-07 10:01:29 +00:00
Zuul
c384c827ec Merge "Remove jenkins-sudo" 2018-02-07 09:53:51 +00:00
Jeremy Stanley
af23479f15 Disable citycloud in nodepool
At the request of Daniel Öhberg, CityNetwork VP of Operations, we
are ceasing booting CI nodes in any remaining regions and pausing
all image uploads for now. We should get back in touch with them
around the beginning of July to find out whether they're ready for
us to reenable use of this account.

Change-Id: I39f12d5a8441b197f5a310c7b33ba42c04c73378
2018-02-06 15:17:11 +00:00
Andreas Jaeger
b72eed6fba Remove zuul-sudo-grep lines from nodepool
File
http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/legacy/pre.yaml#n54
sets up sudoers for zuul-sudo-grep.sh, remove it from nodepool images.

Change-Id: Ibd8b6024975587a0d137477808c212dfd3f91e76
2018-02-05 21:43:38 +01:00
Zuul
b68ad99ab8 Merge "Use zuul-sudo-grep" 2018-02-05 20:35:39 +00:00
Andreas Jaeger
209b27a89b Remove jenkins-sudo
We do not use the jenkins user anymore, so can remove the sudo settings
that allows jenkins to become root.

Change-Id: I69d58c2d10a2f48406cf2991fe83ce0e64851c8c
2018-02-05 21:17:49 +01:00
Mathieu Gagné
dda65fe7b8 Revert "Temporary disable inap-mtl01 region for maintenance"
This reverts commit ed1663c163a4290e21cdb787f3fc9ba71cb34bfa.

Change-Id: I5e4289ac3efc0b7e445b998f2a017f45184a8609
2018-02-01 22:28:37 +00:00
Mathieu Gagné
ed1663c163 Temporary disable inap-mtl01 region for maintenance
Disable for maintenance and debugging.

Change-Id: I6b6eea9f9a8e02ea5df921084e10e9a69c8574a0
2018-02-01 11:43:20 -05:00
Zuul
1e9c383e65 Merge "Remove tripelo-test-cloud-rh1" 2018-01-31 19:20:19 +00:00
Zuul
8812943974 Merge "Remove initialize-urandom from ubuntu-trusty" 2018-01-31 17:22:34 +00:00
Zuul
36ade37958 Merge "Revert "Pause ubuntu-xenial image uploads"" 2018-01-31 16:59:59 +00:00
Zuul
03f5097d8a Merge "Disable tripleo-test-cloud-rh1 for nodepool" 2018-01-31 16:59:56 +00:00
Paul Belanger
45c4ea6f64
Remove initialize-urandom from ubuntu-trusty
During the migration of I4bda502f31087bc125bd452a9075e3caaa031c7d we
incorrectly added initialize-urandom to ubuntu-trusty. This is a
problem because, it only support systemd and only xenial is affected.

Remove to start building DIBs again.

Change-Id: Icd5f7a9ac99abd7c979060e9bea455ea5032f13b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-31 11:13:43 -05:00
Zuul
0e64fc1774 Merge "Revert "Temporarily reduce max-servers in rax-dfw"" 2018-01-31 15:26:27 +00:00
Paul Belanger
949d157a15
Remove tripelo-test-cloud-rh1
The provider is no disabled, remove remaining configuration from
nodepool and grafana.

Change-Id: I61fc4871a407449317efb0d20f44fd62faff1c7a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-31 10:18:00 -05:00
Paul Belanger
ee2253b300
Disable tripleo-test-cloud-rh1 for nodepool
Now that we remove pipelines from zuul, there is no reason to keep
trpielo-test-cloud-rh1 configured in nodepool.

Change-Id: I658fda4fd979065f6783331341f0ad8b03664268
Depends-On: https://review.openstack.org/538023/
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-01-31 10:17:33 -05:00