14133 Commits

Author SHA1 Message Date
Zuul
78c6860192 Merge "run_cloud_launcher.sh : generate runtime stats" 2018-11-09 05:23:02 +00:00
Zuul
5f6dc88264 Merge "bridge.o.o: Use latest openstacksdk" 2018-11-09 02:53:10 +00:00
Zuul
04d00ae988 Merge "Pin bridge.o.o to ansible 2.7.0, add devel testing job" 2018-11-09 02:51:20 +00:00
Jeremy Stanley
2cfb32652a Retire the openstack-internships mailing list
This list is basically unused, and its owners/moderators have asked
for it to be shut down. Once this merges (so that Puppet won't
recreate it), the list can be removed with the `rmlist` utility
(this will still leave the archives available but will remove it
from the list index and no longer accept subscriptions/posts).

Change-Id: Ieaf35b1e6875b26b6c20da8a9e9c2eedf200eaf8
2018-11-08 16:52:25 +00:00
Ian Wienand
06da49c6e2 bridge.o.o: Use latest openstacksdk
Similar to the pinning introduced in
Ic465efb637c0a1eb475f04b0b0e356d8797ecdeb, use the "latest"
openstacksdk package and allow for passing of pinned versions if
required.

Update the devel test to also use the master of opensatcksdk

Change-Id: I4b437ca9024c87903bdd3569c8309cde725ce28e
2018-11-08 09:50:58 +11:00
Ian Wienand
24c81fb0c3 Pin bridge.o.o to ansible 2.7.0, add devel testing job
This adds arguments to "install-ansible" to allow us to specify the
package name and version.

This is used to pin bridge.o.o to 2.7.0 (see
I9cf4baf1b15893f0c677567f5afede0d0234f0b2).

A new job is added to test against the ansible-devel branch. Added as
voting for now, until it proves to be a concern.

Change-Id: Ic465efb637c0a1eb475f04b0b0e356d8797ecdeb
2018-11-08 09:50:53 +11:00
Ian Wienand
e32a9fdd0f run_cloud_launcher.sh : generate runtime stats
Similar to run_all.sh (I299c0ab5dc3dea4841e560d8fb95b8f3e7df89f2),
produce a runtime stat for each run of the cloud launcher.

Although it won't directly highlight errors, problems tend to end this
playbook early.  When graphed with grafana, we could have noticed a
large drop in the average runtime which would have suggested a
problem.

Change-Id: I8e5371cbc94e9a803ea5e64ae94aca293b834c73
2018-11-08 08:43:40 +11:00
David Moreau Simard
e8fd13ae70
Bump amount of mod_wsgi processes for static vhosts to 16
The default of 8 processes does not appear to be sufficient to handle
the concurrent requests for openstack's log analyzer and ara
simultaneously under load.

Change-Id: I928fcbcc670c68bfb7b206949edca1a5e6daf35a
2018-11-07 15:34:47 -05:00
James E. Blair
8c7e21ea9d Remove infracloud from cacti
Change-Id: I7e00cfb32b07024da2d5969351ad429bb9b9d139
2018-11-07 10:20:31 -08:00
Ian Wienand
f295b5b44a Update citycloud project details
These names were taken from the citycloud web interface RC file, but
actually match what we already have in
playbooks/templates/clouds/nodepool_clouds.yaml.j2

Testing with this I can authenticate to openstackzuul-citycloud

Change-Id: Ic7aeb5c3a96e5594b8c9c396daaad7e79c1f5c63
2018-11-07 11:48:17 +11:00
Monty Taylor
cad774e65a
interface is not an auth option
interface indicates which of the public, internal or admin endpoints a
user wants to consume.

Change-Id: I061200bbf4477ab53ec6431c71baa8dda6bea6b5
2018-11-06 15:49:23 -06:00
Colleen Murphy
8498fb0988 Turn on the future parser for wiki.openstack.org
Change-Id: Ie6de2e78b4f8292e9837f628f75ae229637d4c91
2018-11-06 21:27:11 +01:00
Monty Taylor
214662a424 Install latest openstacksdk on bridge
It's designed to always be used from the latest version.

This trips an ansible lint rule (ANSIBLE0010) which we can ignore, as
we often have pip things that we want to install the latest release
of automatically.

Change-Id: Ieac93ab3a555f2423d4fbcf101d6d9681ae0e497
2018-11-07 06:20:33 +11:00
Monty Taylor
20d966a9db
Add Lon1 and Sto2 to openstackzuul cloud entry
These are missing, so cloud launcher is sad.

Change-Id: Ie74259a3e6acc2d6fc6c40ac42e0c80997208f85
2018-11-06 11:19:44 -06:00
Zuul
9679f57225 Merge "Add Arm64 CI cloud" 2018-11-06 01:21:48 +00:00
Colleen Murphy
f3a73fa03f Turn on the future parser for wiki-dev.openstack.org
Change-Id: Id78b81c24b9ac57af27082771146aca7072c0eb6
2018-11-05 22:21:39 +01:00
Colleen Murphy
e9bcd65a5b Turn on the future parser for lists.openstack.org
Change-Id: I9481f1b7cbd0eb1a266e2c0d9ba9d3c784bab737
2018-11-05 22:20:48 +01:00
Colleen Murphy
b45e642f97 Turn on future parser for lists.katacontainers.io
The mailman puppet module is tested and seems to be working on puppet
4[1]. Turn on the future parser for lists.katacontainers.io, which hosts
fewer mailing lists than lists.openstack.org does.

[1] https://review.openstack.org/601464

Change-Id: Iea7fbabf7cee579dcd19d4deb4d7537ed1b13bd9
Depends-On: https://review.openstack.org/601831
2018-11-05 22:18:26 +01:00
Colleen Murphy
e9576c0677 Turn on the future parser for eavesdrop.o.o
eavesdrop.o.o runs a number of different sites and bots, so on the one
hand it is a slightly complex service, but on the other hand these
modules are quite simple and similar to each other. Meetbot, accessbot,
and statusbot have tests and seem to work fine on puppet 4. Ptgbot does
not have tests, but that's why we should try this out sooner rather than
later so we have time to correct any issues before the PTG.

Change-Id: Id1af4b3ebf04f34c381eab5f818815734a296f7b
2018-11-05 22:18:22 +01:00
James E. Blair
d15c6166c3 adns: Set zone directory permissions
Bind needs to be able to write to the zone directories in order
to sign the zones.

Change-Id: I5649c28c6f7d8d98e0eca3c9c4da5d7312198b5c
2018-11-05 09:02:55 -08:00
James E. Blair
dae1a0351c Configure opendev nameservers using ansible
Change-Id: Ie6430053159bf5a09b2c002ad6a4f84334a5bca3
2018-11-02 13:49:38 -07:00
James E. Blair
90e6088881 Configure adns1.opendev.org server via ansible
Change-Id: Ib4d3cd7501a276bff62e3bc0998d93c41f3ab185
2018-11-02 13:49:38 -07:00
Clark Boylan
ab5c1944cc Simplify elasticsearch cluster groups.yaml membership
We run a cluster of elasticsearch nodes which may have additional
servers added to it or old servers replaced. Make this slightly easier
by matching them with a glob in groups.yaml so that we don't have to
edit this file to add new servers.

Change-Id: I8ce3e38f67c545642d561b2afdd6dbbfa677e67f
2018-11-02 12:20:20 -07:00
Clark Boylan
47bf937f8d Simplify git backend server groups.yaml membership
We run many git server backend nodes which may be replaced with new
servers. To simplify the process of adding or replacing git0* servers
stop explicitly listing them in groups.yaml and instead match them on a
glob.

Change-Id: I3dbf814122bbfc508b2a90a5f5a533226fa0bf8f
2018-11-02 12:20:20 -07:00
Clark Boylan
7427cc7a98 Simplify zookeeper groups.yaml membership
We run a cluster of zookeeper nodes. Lets match them with globs in
groups.yaml so that we don't have to edit groups.yaml if expanding the
size ofthe cluster or replacing nodes in the cluster.

Change-Id: Iff9048a9bc15e4f888e7caab54abb4d4a9d340d7
2018-11-02 12:20:20 -07:00
Clark Boylan
b53a03df68 Simplify nodepool groups.yaml membership
Replace all the explicit listings of nb01, nl01, etc with globs so that
we can launch new nodepool builders and nodepool launchers without
editing the groups file.

Change-Id: I3c32fb604109a3c65b2145b733fc0e2545e21045
2018-11-02 12:20:20 -07:00
Clark Boylan
9beb96838b Simplify mirror node listings in groups.yaml
Match the mirrors with a glob so that we can boot new mirrors in new
regions or replacements in existing regions without needing to
explicitly add them to groups.yaml first.

Change-Id: I3e7bf04a2d3a7602e46710840b0cec42c238c6c1
2018-11-02 12:20:20 -07:00
Clark Boylan
1947bfc3ef Simpligy logstash nodes in groups.yaml
Match logstash-worker and logstash.o.o nodes with globs. This allows us
to launch new servers without needing to add them to the groups file
first.

Note that logstash.o.o is currently hosted by logstash01.openstack.org
so the existing groups.yaml content is broken for this host.

Change-Id: Iea9b5815a31355925979e6d4da5c578a64af7972
2018-11-02 12:20:20 -07:00
Clark Boylan
5a98029ccc Cleanup zuul-executors in groups.yaml
Convert ze01, ze02, ze03, etc to ze[0-9]* so that we can launch new zuul
executors as necessary without updating the groups file.

Change-Id: Id9896e89d8bc821de003994a96ae11517a76145d
2018-11-02 12:20:20 -07:00
Clark Boylan
6273236b72 Cleanup zuul-mergers in groups.yaml
The zuul mergers do not run a webserver so we remove them from the
webserver group. Convert zm01, zm02, zm03, etc to zm[0-9]* so that we
can launch new zuul mergers as necessary without updating the groups
file.

Change-Id: Ibe8cbea02990378f4ee5ba7880ee0c8a4e9b7bb9
2018-11-02 12:20:20 -07:00
Clark Boylan
6096e0cf28 Remove mirrors from webservers in groups test
We removed the mirror nodes from the webservers group to fix iptables
rule application on the nodes. Unfortunately we didn't update our test
that tries to assert mirrors should be in the webservers group. Update
the test results fixture to remove webservers as a valid group for a
mirror node.

Change-Id: Iba18e54f4df4a36c0247f65642faacca9d195769
2018-11-02 12:20:20 -07:00
Jeremy Stanley
681574b352 Merge "Update etherpad group membership" 2018-11-02 18:12:01 +00:00
Jeremy Stanley
d874d4da63 Remove mirrors from the webservers group
The webservers group is meant to be mutually exclusive to any other
iptables modifications but was inadvertently expanded to include the
mirror servers by Iea8b2eb2e9c723aca06f75d3d3307893e320cced. Remove
the glob as it should not be there and is overriding the
mirror-specific firewall rules.

Change-Id: Ia1b702e5c9a2d433536edcec4bfaf6f47b1c0291
2018-11-02 18:02:43 +00:00
Clark Boylan
855e9cf5e0 Update etherpad group membership
Our etherpad servers now have digits so we need them to have digits in
the groups file. Also we have etherpad-dev group_vars that are different
than the etherpad group_vars values so add a group for etherpad-dev to
ensure the correct hiera data is used on the dev server.

Change-Id: I9e34ef7969b8a562192a29d98ead6fc55f71d5f2
2018-11-02 10:19:37 -07:00
Zuul
015eec18f3 Merge "Remove puppet config for opendev nameservers" 2018-11-02 15:54:28 +00:00
Zuul
2446e616d0 Merge "Add unittest for yamlgroup inventory plugin" 2018-11-02 15:47:35 +00:00
Zuul
e6f2ad0d8b Merge "Add yamlgroup inventory plugin" 2018-11-02 15:46:09 +00:00
Zuul
e8314158fc Merge "Make the pip3 role really install something" 2018-11-02 14:21:27 +00:00
Gary Perkins
10c8da0e90 Add Arm64 CI cloud
Changes:
  * Set URLs to https://
  * Set new keystone domain
  * Set region to nrt1

Change-Id: Ib96d14008ab3b8b7c12429d7432eaa485c404bb2
2018-11-02 12:22:15 +00:00
Zuul
c1a1b56278 Merge "Install current ansible" 2018-11-01 22:10:16 +00:00
James E. Blair
61429518f2 Remove puppet config for opendev nameservers
These will be configured with ansible since the hosts run on
bionic.

Change-Id: I102eb6be179d84ae308d2a3cadd523bdfb2d8601
2018-11-01 14:22:45 -07:00
Ian Wienand
526a423dd2 Add unittest for yamlgroup inventory plugin
This mocks out enough of the Ansible inventory framework so we can
test the group matching against a range of corner cases as present in
the results.yaml file.

Change-Id: I05114d9aae6f149122da20f239c8b3546bc140bc
2018-11-02 08:20:02 +11:00
Monty Taylor
e998db36f2 Add yamlgroup inventory plugin
The constructed inventory plugin allows expressing additional groups,
but it's too heavy weight for our needs. Additionally, it is a full
inventory plugin that will add hosts to the inventory if they don't
exist.

What we want instead is something that will associate existing hosts
(that would have come from another source) with groups.

This also switches to using emergency.yaml instead of emergency, which
uses the same format.

We add an extra groups file for gate testing to ensure the CI nodes
get puppet installed.

Change-Id: Iea8b2eb2e9c723aca06f75d3d3307893e320cced
2018-11-02 08:19:53 +11:00
Zuul
c2b863cff1 Merge "Nodepool.o.o is no longer a thing, remove it" 2018-11-01 20:36:34 +00:00
Clark Boylan
0f8e7a91bb Nodepool.o.o is no longer a thing, remove it
We've only been using nodepool.o.o as a zookeeper server for the past
year or so. Last week we transitioned to a three node zookeeper cluster
and stopped using nodepool.o.o. This server has since been deleted.

This is the last bit of cleanup to remove it from config management.

Change-Id: I9d0363393ed20ee59f40b210ea14fb105a492e20
2018-11-01 12:09:08 -07:00
Zuul
6ac467caa4 Merge "Set ansible python version for opendev nameservers" 2018-11-01 17:44:21 +00:00
James E. Blair
2780973330 Set ansible python version for opendev nameservers
Change-Id: Icaee291e872b6a19793a1ba003e55a43f3898ea7
2018-11-01 09:20:26 -07:00
Zuul
26494b03d9 Merge "Hyperlink task footers" 2018-10-31 19:13:19 +00:00
Zuul
c488c48a9e Merge "Add opendev nameservers (2/2)" 2018-10-31 16:35:52 +00:00
Jens Harbott
fc2e7dbe01 Make the pip3 role really install something
It seems that the package module doesn't have a default value for the
state of a package. So not specifying it is essentially a no-op. Specify
state: present so that we really get the packages installed that we want
to have.

Change-Id: Ied9484c74922b9f48e14205b73ca95066a68a87f
2018-10-31 15:21:33 +00:00