1563 Commits

Author SHA1 Message Date
Clark Boylan
bde73bb690 Stop running unnecessary tests on trusty
We tag our site.pp with annotations that indicate if certain hosts
should be testedo ntrusty or not. We've semi recently upgraded several
services that no longer need to be tested on trusty so we remove them
here.

Change-Id: I79e6e1e555eca9e93acb37303a80cecae7d8d92d
2018-12-14 14:02:14 -08:00
Zuul
bec9e766eb Merge "Serve opendev.org website from files.o.o" 2018-11-30 18:38:47 +00:00
James E. Blair
5645c0da86 Enable relative_priority in zuul
Change-Id: I23f413e13955977fac66018ef6492d7cf0475751
2018-11-30 08:42:35 -08:00
James E. Blair
a14ba7c9a5 Serve opendev.org website from files.o.o
Change-Id: I8e4bd3a66fbafdef2644da0719d7bcbe6ae92666
2018-11-29 10:13:45 -08:00
Ian Wienand
85297878f5 Retire bandersnatch mirroring
Bandersnatch mirroring has been disabled since
I88a838cb28fee3bd16b2b0a26e614ac5c2f23241 which is currently almost 6
months ago.  Since then we have been running a reverse caching proxy.

Although bandersnatch served us well, it seems pypi has become
impractical to mirror locally.  This is partially due to 2TB volume
limitations of OpenAFS and partially due to us not having a sane way
to filter large, frequently updating packages.  With the reverse proxy
working there are no plans to restore our local mirror.

Retire the references to it before we clean up the AFS volumes.

Change-Id: Ia23828328dd859bbf26f95735c1c2e99c573d10e
2018-11-16 10:28:11 +11:00
James E. Blair
61429518f2 Remove puppet config for opendev nameservers
These will be configured with ansible since the hosts run on
bionic.

Change-Id: I102eb6be179d84ae308d2a3cadd523bdfb2d8601
2018-11-01 14:22:45 -07:00
Clark Boylan
0f8e7a91bb Nodepool.o.o is no longer a thing, remove it
We've only been using nodepool.o.o as a zookeeper server for the past
year or so. Last week we transitioned to a three node zookeeper cluster
and stopped using nodepool.o.o. This server has since been deleted.

This is the last bit of cleanup to remove it from config management.

Change-Id: I9d0363393ed20ee59f40b210ea14fb105a492e20
2018-11-01 12:09:08 -07:00
Zuul
c488c48a9e Merge "Add opendev nameservers (2/2)" 2018-10-31 16:35:52 +00:00
Zuul
6ae47aebd9 Merge "Switch zuul scheduler to new zk cluster" 2018-10-25 14:36:05 +00:00
James E. Blair
4712bdf3f7 Add opendev nameservers (2/2)
This uses IP addresses to bootstrap serving opendev.org on these
servers.

Change-Id: I769097192583c4507b19b914e4c75c60948de1bf
2018-10-25 06:27:21 -07:00
Clark Boylan
2c077aa2d3 Fix zk cluster members listing
First we convert from a hash to an array to list the zk cluster members.
Do this because the older puppet-zookeeper module we use doesn't support
hashes like newer releases do.

Second we convert from hostnames to IP addresses because the hostnames
were looked up in /etc/hosts to resolve to 127.0.1.1 which caused zk to
listen on 127.0.1.1 and the cluster members could not talk to each
other.

Change-Id: Id590eb535301b27ff0dcb27cf65ba85d61ed19a5
2018-10-22 16:22:03 -07:00
Clark Boylan
c8e0df2dc2 Configure zk\d to act as a cluster
You have to tell the zk servers what servers belong to the cluster to
have them act as a cluster. Currently all our servers are in mode
standalone as they do not know about the other servers.

Fix this by configuring the three servers as servers 1, 2, and 3 in the
zk config.

Change-Id: I985ac7387744498058f99859eb8445f885b8bbcc
2018-10-22 09:50:55 -07:00
Clark Boylan
0d502ae724 Switch zuul scheduler to new zk cluster
Switch the zuul scheduler to use our new zk cluster. Note this should
happen after the builders have switched and populated image data on the
new database cluster and at the same time as the launchers update.

This gets us off the old single node zk server running on trusty to
newer xenial based zk cluster with HA.

This change does update the config for mergers and executors for
consistency, but I believe that only the scheduler makes use of this
config today.

Change-Id: I132c176af188787cec3a5a562ffdc81f818c26b0
Depends-On: https://review.openstack.org/612441
2018-10-22 09:33:28 -07:00
Zuul
015e82e1eb Merge "Add opendev nameservers (1/2)" 2018-10-17 17:41:58 +00:00
James E. Blair
f5d3ee4b79 Add opendev nameservers (1/2)
This will allow us to create new nameservers in the opendev.org
domain.  We will replace the existing servers once these are
bootstrapped.

Some lines are commented pending server creation.

Change-Id: If71e3f87a9d7a83d80cff053874c84411b248515
2018-10-16 14:40:10 -07:00
Zuul
cab657cf65 Merge "Add digit support to etherpad servers" 2018-10-08 16:52:28 +00:00
Zuul
0027336f45 Merge "Add docs.starlingx.io vhost" 2018-10-02 20:05:23 +00:00
Clark Boylan
0f0031d16e Add digit support to etherpad servers
Looking at upgrading etherpad servers to Xenial and realize that we
haven't "digitized" them yet. Lets get that in first so that the new
instances we boot can be etherpad01 and etherpad-dev01.

Change-Id: I700a1d018ab37436f11b2d883d79c82db7f1d6b1
2018-09-24 13:33:44 -07:00
Ian Wienand
fa298b2467 Remove puppet nodepool credentials
Since the referenced change in the comment, these values are no longer
necessary.

Change-Id: Ia8490bb0c6e4a71a2809fdfbefb6d12e3d89612f
2018-09-11 09:40:04 +10:00
Dean Troyer
ecd3357790 Add docs.starlingx.io vhost
Change-Id: I91e773bc1b4bd49f855d1b0acee673d70d51e4ba
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-09-10 11:37:30 -05:00
Clark Boylan
09288c7c37 Manage clouds.yaml files in ansible
This manages the clouds.yaml files in ansible so that we can get them
updated automatically on bridge.openstack.org (which does not puppet).

Co-Authored-By: James E. Blair <jeblair@redhat.com>
Depends-On: https://review.openstack.org/598378
Change-Id: I2071f2593f57024bc985e18eaf1ffbf6f3d38140
2018-09-04 08:49:00 -07:00
Monty Taylor
eb086094a8 Install limestone CA on hosts using openstacksdk
In order to talk to limestone clouds we need to configure a custom CA.
Do this in ansible instead of puppet.

A followup should add writing out clouds.yaml files.

Change-Id: I355df1efb31feb31e039040da4ca6088ea632b7e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-31 12:17:35 -07:00
Monty Taylor
15663daaf7 Add iptables role
Co-Authored-By: James E. Blair <corvus@inaugust.com>
Change-Id: Id8b347483affd710759f9b225bfadb3ce851333c
Depends-On: https://review.openstack.org/596503
2018-08-27 14:33:32 +00:00
Monty Taylor
c4b111691b
Install ssh key on bridge.openstack.org
We copied this over from puppetmaster, but let's manage it in ansible.

The key has been renamed in host_vars on bridge.openstack.org already.

Change-Id: Ia102dbe2ae2836880092b8997cb99135f5197b00
2018-08-19 10:26:10 -05:00
Monty Taylor
c716240692
Clean up puppetmaster puppet config handled by ansible
We have a bunch of this handled now in ansible, so remove the old stuff.

Remove puppetmaster group management files. It's confusing for there to
be two files. Remove the old one.

Remove mqtt config. This isn't really a thing currently, and we're
eyeing running things from zuul anyway, so no need to port to ansible.

Change-Id: I8b64d21eadcc4a08bd5e5440fc5f756ae5bcd46b
2018-08-17 11:53:52 -05:00
Monty Taylor
bab6fcad3c
Remove base.yaml things from openstack_project::server
Now that we've got base server stuff rewritten in ansible, remove the
old puppet versions.

Depends-On: https://review.openstack.org/588326
Change-Id: I5c82fe6fd25b9ddaa77747db377ffa7e8bf23c7b
2018-08-16 17:25:10 -05:00
Zuul
983fa5269a Merge "Cleanup nodepool.openstack.org to be a zk server" 2018-08-13 23:00:52 +00:00
Zuul
4c3997d948 Merge "Use nodepool-builder.logging.conf from puppet-nodepool" 2018-08-13 22:32:15 +00:00
Zuul
8f8d0ac127 Merge "Remove references to long-gone odsreg.o.o service" 2018-08-13 16:53:51 +00:00
Zuul
06602fbdcf Merge "Remove references to stackalytics.o.o" 2018-08-09 08:50:58 +00:00
Colleen Murphy
46ebeeb4fb Remove references to stackalytics.o.o
stackalytics.openstack.org does not resolve and seems very dead. Remove
its node from site.pp and remove it from the docs to avoid confusion
about what servers we're really managing. We can always add it back when
the time comes to try again.

Change-Id: I733130ebe97ae7e06ca57b3c8e3a8708fcfa069c
2018-08-07 22:12:43 +02:00
Jeremy Stanley
30f3039f1e Remove references to long-gone odsreg.o.o service
The odsreg.openstack.org server was removed from service 2018-01-11,
and should also be cleared from our global site manifest. It was the
only thing utilizing the puppet-odsreg module, so we can take that
out of the modules list as well (that will be retired separately).

Change-Id: Iadfddb3bf57428b928cacaaa672e24c4a1e92058
2018-08-07 19:52:13 +00:00
Ian Wienand
cbff37f82e Cleanup nodepool.openstack.org to be a zk server
This is just a zookeeper server now, so doesn't need all these
passwords, a cloud.yaml file or a launcher-esque deployment.

Change-Id: I3cb20d9c8af150ecdc1fb0a16208a774e3fb530f
2018-07-30 14:12:33 +10:00
David Shrewsbury
10e4c57128 Use nodepool-builder.logging.conf from puppet-nodepool
Generating a openstack-infra specific version of this file was
deprecated with change I6fdc3b622454b069f3ad2cf42da584d8df23a110 when
build logs was moved out of Python logging.

This means (post the depends-on) the file is exactly as kept in
puppet-nodepool.  Remove the system-config template and just use that.

Depends-On: https://review.openstack.org/586231
Change-Id: I2db37868be12c146678f1c8934db80050c788540
2018-07-30 14:07:53 +10:00
James E. Blair
a1163f9952 Fix syntax of gerrit host ssh keys
We need to wrap the hostnames/ips with []s and supply the port numbers
for the files to be effective with paramiko.

Change-Id: Iab12951a828b7c27ef2255137a6bc0b69ca3a770
2018-07-27 14:38:34 -07:00
James E. Blair
6402f98d12 Add opendaylight connection to zuul
This adds a connection to the opendaylight Gerrit server to zuul
so that we can perform cross-community testing.

Change-Id: I846983e873072c2235a4c49e36f602a47d06cfce
2018-07-26 13:56:15 -07:00
Paul Belanger
55b04c9ce9
Bring ze11.o.o online
This brings online our 11th zuul-executor, and opens the required
firewall ports.

Change-Id: If0ee569a2d14caeeb912b7382160f47d460650ab
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-07-20 16:18:37 -04:00
Zuul
0fb4948eb6 Merge "Add git.starlingx.io" 2018-06-27 21:30:28 +00:00
Clark Boylan
8f9db69c81 Don't use snakeoil files with real certs
The refstack puppetry is set up to use snakeoil certs by default which
is great for bootstrapping; however, our production instance has real
certificate issued to it and we shouldn't write that to the snakeoil
cerrt files to avoid confusion.

Specificy refstack specific certificate, key, and intermediate chain
file locations to clean this up and allow us to revert our snakeoil
certs to proper untrusted self signed certs.

Change-Id: Ibbcdd93a2ae38d9063b1f88f90ebdaadcac9b585
2018-06-22 15:38:37 -07:00
Zuul
d8e1ad6b87 Merge "Template credentials for Packet Host/Platform 9" 2018-06-12 16:20:57 +00:00
Jeremy Stanley
acb2e9e6f0 Template credentials for Packet Host/Platform 9
Packet Host and Platform 9 have generously agreed to donate some
compute resources to our testing efforts. Add Nodepool and
Puppetmaster credentials for them.

Change-Id: I705c4204abca060c35a1a417791a67229b78cd02
2018-06-12 00:59:09 +00:00
Clark Boylan
73aadfd49f Apply up to date gerrit host key to zuul servers
Apply the review.o.o ssh rsa host key to all of our active zuul servers.
Update the ip addresses in that host key entry to reflect the current
server's IP addrs too.

Change-Id: I147ff0b9547f2cee36d7c56c8f5352ece8a4ec82
2018-06-07 11:56:48 -07:00
Clark Boylan
8304b9626c Revert "Add opendaylight connection to zuul"
This reverts commit c7bb14bba6ec00fff61879154a3f5f143c98767b.

SSH connectivity isn't quite working yet. Revert until we can get that
sorted out.

Change-Id: If2a88a61ed592f927980c71486ed0b7e3cb848d0
2018-06-07 08:47:19 -07:00
Dean Troyer
6a912d2787 Add git.starlingx.io
Change-Id: Ie54b0cb24a7b91ba64add6177ba6d2c167fdb601
2018-06-06 09:42:51 -05:00
Zuul
f704863313 Merge "Add opendaylight connection to zuul" 2018-06-05 19:21:42 +00:00
Ian Wienand
0b21d2f61a Allow mirror-update.o.o to produce stats
Open the firewall port for mirror-update so we can start logging some
stats about AFS partition and volume usage, initially for a grafana
dashboard.

Change-Id: I361e7213ed4b4ed4d3fcc8fdbee06e2fe677934a
2018-06-01 17:09:48 +10:00
Zuul
eea6c0e468 Merge "This patch impliments the survey spec: I3c389596373b94459a32a4e540d514a2941acbb1" 2018-05-31 19:46:25 +00:00
James E. Blair
c7bb14bba6 Add opendaylight connection to zuul
This adds a connection to the opendaylight Gerrit server to zuul
so that we can perform cross-community testing.

Change-Id: I840b24ebe52c50840180f2dda40a3f8669baa347
2018-05-31 16:43:51 +00:00
Zuul
35655d20d0 Merge "Open port 443 for builder logs" 2018-05-21 08:19:13 +00:00
Ian Wienand
0896b902de Open port 443 for builder logs
Follow-on to Ifd5823ff51de40226c72930c69faf50b677a7dfe to open the
port for SSL.

Change-Id: Iceeb04c473baaf6d90b63aa066bf18ee30c7b119
2018-05-21 17:07:16 +10:00