Commit Graph

13799 Commits

Author SHA1 Message Date
Ian Wienand
86adc42d5c Add ansible runtime script
This is designed to run on bridge.o.o and give us an overview of the
last few ansible cron runs so we can see if there are issues.

Change-Id: I1b23cac74272af891d0b29963dc943bd54128664
2018-09-05 17:19:39 +10:00
Zuul
e54afccc77 Merge "Manage clouds.yaml files in ansible" 2018-09-05 02:26:12 +00:00
Zuul
ae24516086 Merge "Add logrotate role and rotate ansible log files" 2018-09-05 00:33:44 +00:00
Zuul
4958f710d3 Merge "Set Ansible forks to 50" 2018-09-05 00:01:51 +00:00
Ian Wienand
3657cacfca Add logrotate role and rotate ansible log files
Add a logrotate role that allows basic configuration of a logrotate
configuration for a specific log-file.

Use this role in the ansible-cron and install-ansible roles to ensure
the log output they are generating is rotated.

This role is not intended to manage the logrotate package (mostly to
avoid the overhead of frequently checking package state when this is
expected to be called for multiple configuration files on a server).
We add it as a base package to our servers.

Tests are added for testinfra.

Change-Id: I90f59c3e42c1135d6be120de38e942ece608b761
2018-09-05 09:15:46 +10:00
Zuul
5cb8af4744 Merge "Install limestone CA on hosts using openstacksdk" 2018-09-04 23:05:35 +00:00
James E. Blair
8419909571 Set Ansible forks to 50
20 is working fine with plenty of ram/cpu to spare, increase to 50
to attempt to speed up the runtime.

The environment variable should be used by default, but the "-f"
option will override that, in the one case where we need it.

Change-Id: Ie6a1d991a346702ec58cd716b0b94af5c93554ac
2018-09-04 14:15:48 -07:00
Paul Belanger
a28875c4cc Use package task over apt for base-server
Package is the generic way of using package managers in Ansible. This
will be a noop.

Don't use loops for package managers, since we are able to pass lists of
packages. This will reduce the number of tasks ansible will do.

Change-Id: If7988ba81a6bf851d1b5ec9db6888ba9509ed788
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-09-04 15:21:48 -04:00
Clark Boylan
09288c7c37 Manage clouds.yaml files in ansible
This manages the clouds.yaml files in ansible so that we can get them
updated automatically on bridge.openstack.org (which does not puppet).

Co-Authored-By: James E. Blair <jeblair@redhat.com>
Depends-On: https://review.openstack.org/598378
Change-Id: I2071f2593f57024bc985e18eaf1ffbf6f3d38140
2018-09-04 08:49:00 -07:00
Clark Boylan
24c67ecb0e Run puppet on mirror02 in packethost
We have made some mirror config changes that are not being deployed here
because we have disabled puppet on this node. I do not think we need to
disable puppet here so go ahead and reenable it.

Change-Id: If7da76d24ea64db3c038bc60f64fa39a4f5f6f72
2018-09-03 09:03:40 -07:00
Monty Taylor
eb086094a8 Install limestone CA on hosts using openstacksdk
In order to talk to limestone clouds we need to configure a custom CA.
Do this in ansible instead of puppet.

A followup should add writing out clouds.yaml files.

Change-Id: I355df1efb31feb31e039040da4ca6088ea632b7e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-31 12:17:35 -07:00
Zuul
3d29a8114d Merge "Remove releasestatus configuration" 2018-08-31 16:49:05 +00:00
Zuul
40b8e6691b Merge "Remove obsolete pypi-mirror" 2018-08-31 16:49:04 +00:00
Zuul
21a81de59f Merge "Don't use loops with package task" 2018-08-30 20:53:32 +00:00
Zuul
c7f2cca1fa Merge "unbound: fix dhclient.conf copy" 2018-08-30 20:38:08 +00:00
Zuul
017fdf3e34 Merge "run-base: exclude __pycache__ from logs" 2018-08-30 18:57:02 +00:00
Zuul
7c2b6442c2 Merge "Refactor run-base jobs" 2018-08-30 18:48:20 +00:00
Zuul
2a51a493e0 Merge "Add system-config-run-eavesdrop" 2018-08-30 18:38:18 +00:00
Zuul
94ea02f635 Merge "run-base: collect ansible configuration" 2018-08-30 18:29:56 +00:00
James E. Blair
a491f80f6f run-base: exclude __pycache__ from logs
We don't need to copy this back over.

Change-Id: I419c9c80526953177f808ad7b180deba9bf0434e
2018-08-30 11:05:49 -07:00
Zuul
768df606b0 Merge "Generate junit.xml file for testinfra" 2018-08-30 17:58:39 +00:00
James E. Blair
6dab882147 Refactor run-base jobs
Create a parent run job and inherit from it.  This reduces duplicate
'run' parameters, and corrects the omission of run-post from the
eavesdrop job.

Change-Id: Ib2a21b7190bf3611972097d6db545989cd54b3d4
2018-08-30 10:48:58 -07:00
James E. Blair
09b1ff4bc3 Add system-config-run-eavesdrop
Add a job which runs testinfra for the eavesdrop server.  When we
have a per-hostgroup playbook, we will add it to this job too.

The puppet group is removed from the run-base job because the
groups.yaml file is now used to construct groups (as it does
in production) and will construct the group correctly.

The testinfra iptables module may throw an error if it's run
multiple times simultaneously on the same host.  To avoid this,
stop using parallel execution.

Change-Id: I1a7bab5c14b0da22393ab568000d0921c28675aa
2018-08-30 10:25:23 -07:00
James E. Blair
6de5aee53b run-base: collect ansible configuration
This is helpful for debugging.

Change-Id: I0423dcede48c98379e58edd245842afe3292c0b1
2018-08-30 10:25:19 -07:00
Zuul
8a7d25d7ed Merge "Install python2.7 on xenial nodes" 2018-08-30 16:57:27 +00:00
Zuul
0980cafc11 Merge "Update etherpad.o.o to etherpad-lite 1.7.0" 2018-08-29 22:47:31 +00:00
Zuul
ba3e164b1e Merge "Unlink proper path to ansible cache" 2018-08-29 19:21:02 +00:00
Zuul
37adace72b Merge "Use python3 and modern ansible for launch node" 2018-08-29 19:21:01 +00:00
Zuul
40a17586cc Merge "Enable cbs.centos.org mirror" 2018-08-29 18:53:05 +00:00
Zuul
0cf7a13202 Merge "base-test: iptables: allow zuul console streaming" 2018-08-29 18:19:36 +00:00
James E. Blair
800397c3da base-test: iptables: allow zuul console streaming
This adds a group var which should normally be the empty list but
can be overridden by the test framework to inject additional iptables
rules.  It's used to add the zuul console streaming port.  To
accomplish this, the base+extras pattern is adopted for
iptables public tcp/udp ports.  This means all host/group vars should
use the "extra" form of the variable rather than the actual variable
defined by the role.

Change-Id: I33fe2b7de4a4ba79c25c0fb41a00e3437cee5463
2018-08-29 09:20:42 -07:00
Paul Belanger
20286235b6 Generate junit.xml file for testinfra
And collect it on post, it is helpful to see the results.

Change-Id: I0dbecf57bf9182168eb6f99cdf88329fcdeb1bdc
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-29 12:00:43 -04:00
Jeremy Stanley
db5a65332b Update etherpad.o.o to etherpad-lite 1.7.0
Switch to etherpad-lite 1.7.0 in production prior to the PTG. This
version brings a couple more security fixes since 1.6.6 and also
fixes the bullet list authorship misattribution bug. See
https://github.com/ether/etherpad-lite/blob/1.7.0/CHANGELOG.md for a
summary of changes. The version of NodeJS we're using now should
meet the minimum requirements (but will need to be updated prior to
the next etherpad-lite release). This version is the one currently
served from etherpad-dev.openstack.org and can be tested there as
desired.

Change-Id: If52d1b1c3dc33da56133ccb5e6adf33ebd3d2428
2018-08-29 14:22:13 +00:00
Andreas Jaeger
207d4dc828 Remove releasestatus configuration
releasestatus is not used since December 2015, see
https://review.openstack.org/#/c/254817/ .

We can remove the disablement of releasestatus now.

Change-Id: I0c8f5f45463d563dcd6aff8135cc86091e0aa19c
2018-08-29 12:25:49 +02:00
Andreas Jaeger
c5a9e3793b Remove obsolete pypi-mirror
We do not use pypi-mirror anymore, there's also no usage of pypimirror
in jeepyb. Remove the now obsolete module.

Related change: https://review.openstack.org/597370/
Change-Id: I13423bf55eac57da18449852e2102c9633d595bb
2018-08-29 09:00:36 +02:00
Zuul
9958b253f3 Merge "Don't git clone system-config in puppet-install" 2018-08-29 00:20:53 +00:00
Paul Belanger
17a8a70643 Don't git clone system-config in puppet-install
Talking to clarkb, it was decided we can remove this logic in favor of
having ansible-role-puppet push system-config and modules to the remote
nodes.

Change-Id: I59b8a713cdf2b4c1fede44e977c49be5e8cc08fa
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-28 19:28:48 -04:00
Paul Belanger
30c2e03281 Don't use loops with package task
We can directly pass a list of packages to the package task in ansible,
this will help save us some times on run times.

Change-Id: I9b26f4f4f9731dc7d32186584620f1cec04b7a81
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-28 18:32:42 -04:00
Paul Belanger
ad0e8bd241 Add profile_tasks and timer callbacks for ansible
These add extra datetime information to tasks, in an effort to help
profile ansible. More info at:

  https://docs.ansible.com/ansible/2.5/plugins/callback/profile_tasks.html
  https://docs.ansible.com/ansible/2.5/plugins/callback/timer.html

Change-Id: Iebd40bfe06484ec1c6f938570eb6cb60e532fb9f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-28 18:30:31 -04:00
Zuul
62a74b0630 Merge "Test puppet is installed on base" 2018-08-28 22:21:13 +00:00
Zuul
65f96790cb Merge "Add install-puppet to base playbook" 2018-08-28 22:18:45 +00:00
Zuul
ca5f61ecb5 Merge "allow cr repo mirror for early testing" 2018-08-28 22:10:33 +00:00
James E. Blair
c6b22d7e13 unbound: fix dhclient.conf copy
The original version of this was wishful thinking: "is file" only
works locally, but this needs to run on the remote node.

Change-Id: Ib683809fdf580f41d213308331925c4765bb09d9
2018-08-28 14:28:48 -07:00
Clark Boylan
60d31fc636 Test puppet is installed on base
We need to be able to install puppet in our base ansible as part of the
transition from puppet to other management. Test using testinfra that
our base ansible playbook does install puppet.

Change-Id: I3a080a0717483a0885fefb329a168dd438eb9854
2018-08-28 14:21:08 -07:00
Zuul
70730c0fe3 Merge "Create ansible roles to install puppet" 2018-08-28 20:26:57 +00:00
Clark Boylan
f48c8d9bac Install python2.7 on xenial nodes
Ubuntu xenial does not come with python2 by default. In order to
accomodate a transition from trusty nodes to xenial nodes that are
managed by ansible we want to use python2 on trusty and xenial. Then
when a group of nodes are fully xenialed we can force ansible to use
python3 instead.

Eventually we will have no trusty nodes and can default to using
python3 instead and just have to have a small number of exceptions for
centos.

Change-Id: If1d97e25069d6ed5012c147024aad4d921febfc8
2018-08-28 13:08:45 -07:00
Zuul
f430b70fd1 Merge "Correct inventory pattern for planet.openstack.org" 2018-08-28 19:48:40 +00:00
Zuul
c6e73b8cb6 Merge "Enable unattended upgrades" 2018-08-28 19:33:11 +00:00
Jeremy Stanley
f119cc5aab Correct inventory pattern for planet.openstack.org
The planet.openstack.org service is currently provided by the
planet01.openstack.org server, so correct the inventory pattern for
the webservers group accordingly. This was spotted as the firewall
rules for planet ceased allowing HTTP/HTTPS connections.

Change-Id: I4e3353e8f1a73db13c54dfe93a6a26fc618d2aa4
2018-08-28 19:01:19 +00:00
Zuul
c5c32ed8f4 Merge "Only run role integration tests on role changes" 2018-08-28 17:16:54 +00:00