14886 Commits

Author SHA1 Message Date
Ian Wienand
b6c3c2eb68 Pin ARA on devel job to stable branch
As noted in the linked thread, we need to stay on the stable branch
until we update various bits for the 1.0 version of ARA.  This should
fix the -devel job.

Change-Id: I3b5931cc9b8d55feb66971daed1ef28621da4b59
2019-06-11 18:06:45 +10:00
Zuul
9867d6c6bb Merge "Update to ansible-lint 4.1.0" 2019-06-11 01:48:18 +00:00
Zuul
88909d0a20 Merge "bridge.opendev.org: use Ansible 2.8.0 stable" 2019-06-11 01:48:16 +00:00
Clark Boylan
1884a22ca7 Update certcheck domains
This removes groups.openstack.org as this service was shut down. Add new
opendev services behind ssl.

Change-Id: I14c667c8fbde07c3a52778bc2c5e93abf8f053a4
2019-06-10 13:03:35 -07:00
Jeremy Stanley
d0ff3e48d1 Suppress progress for git gc cron on Gitea servers
The stdout progress feed from `git gc` is fairly verbose and
targeted at audiences running it interactively. Since our cron for
this iterates over thoudands of repositories on our Gitea servers,
we don't need to send the progress info to all our sysadmins by
E-mail. Instead use the --quiet option to the gc subcommand so that
progress output will be suppressed.

If this still proves too verbose (as in, continues to result in
E-mail to root even when there are no failures), we can try
redirecting stdout to /dev/null.

Change-Id: Idc06e48cbf85e127a343c2a3cf51a35e6ed09685
2019-06-09 14:30:28 +00:00
James E. Blair
3199e3b225 Enable SPF checking on lists
This requires an external program and only works on Debian hosts.

Newer versions of exim (4.91) have SPF functionality built-in, but
they are not yet available to us.

Change-Id: Idfe6bfa5a404b61c8761aa1bfa2212e4b4e32be9
2019-06-07 10:34:33 -07:00
Zuul
a12de2104e Merge "mirror: rename 80/443 log files" 2019-06-07 13:21:00 +00:00
Ian Wienand
42e54e2c08 mirror: rename 80/443 log files
Having proxy_[80|443]_access.log is wrong beacuse they're not really
proxies (I think I just copied this incorrectly).  Change it to
mirror_, and update the macro that is only used on the mirror portions
too.

Change-Id: I8eca941fee9606d25dd25bc54bc552ccc7094e0f
2019-06-07 10:14:14 +10:00
Clark Boylan
fd97e061ae Evaluate files vhosts after we determine ssl file paths
Previously we evaluated the vhost templates before setting
ssl_cert_file_ and ssl_key_file_ and ssl_chain_file_. This made erb
unhappy because those are the three variables we use to set paths in the
vhost. Fix this by moving the vhost after the ssl file vars are set.

Change-Id: I4ba62521c9e7da104f8799d016cbcf0214cbdfc1
2019-06-06 16:28:37 -07:00
Ian Wienand
52780440ff Update to ansible-lint 4.1.0
In a follow-on change (I9bf74df351e056791ed817180436617048224d2c) I
want to use #noqa to ignore an ansible-lint rule on a task; however
emperical testing shows that it doesn't work with 3.5.1.  Upgrading to
4.1.0 it seems whatever was wrong has been fixed.

This, however, requires upgrading to 4.1.0.

I've been through the errors ... the comments inline I think justify
what has been turned off.  The two legitimate variable space issues I
have rolled into this change; all other hits were false positives as
described.

Change-Id: I7752648aa2d1728749390cf4f38459c1032c0877
2019-06-06 22:13:12 +00:00
Clark Boylan
9ea8edc341 Evaluate files website vhosts in context of website not vhost
To deal with puppet scoping fun we evaluate the template for our
files.o.o website vhosts in the context of the website define and not in
the context of httpd::vhost.

Change-Id: I90bb881eb6ad78cede3a8a2548e1dfcf24e1160b
2019-06-06 15:12:15 -07:00
Zuul
0ace6c2614 Merge "Add tarballs.opendev.org vhost" 2019-06-06 20:03:33 +00:00
Zuul
f25deabf9c Merge "Add db backups to gitea" 2019-06-06 19:38:51 +00:00
Ian Wienand
8a06d48c84 bridge.opendev.org: use Ansible 2.8.0 stable
Testinfra works with Ansible 2.8.0 now, so we can update
bridge.opendev.org to the latest version.  This also needs an ARA
update; bring it to the latest 0.16.4 release.

Update test-requirements so that tox/ansible-lint use Ansible 2.8.0
too.  See note inline about dependencies.

Note we replace import_tasks with include_tasks in handlers to address
this porting issue:
https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.8.html#imports-as-handlers

Change-Id: I7ed75d253857f86b68f67023af6897af4e1b4f50
2019-06-06 11:25:06 -07:00
Zuul
3e148d0219 Merge "Get an LE cert for tarballs.opendev.org" 2019-06-06 14:53:30 +00:00
Ian Wienand
6b9907dcd8 Add centos/ to centos mirror path
As a follow-on to I0e110ef51c8ed301fd8280ae7fc039e3b01db92c; this
dropped the /centos/ from the base mirror, add it back.

Also switch the mirror to the only one on the altarch-mirrors page
that is in US/TX, which from the name is in Dallas, which must be
pretty close to rax.dfw where the update server lives.

Change-Id: If4d71865f4328e73a26c7b38300767ed6b790579
2019-06-06 13:36:36 +10:00
Zuul
8b98aff89c Merge "centos-mirror-update: handle non-x86 architectures" 2019-06-06 00:57:09 +00:00
James E. Blair
3c84c65614 Add tarballs.opendev.org vhost
Change-Id: I6aa85bf92b2d5726d3c86b11b103a87f11953c51
2019-06-05 14:10:19 -07:00
James E. Blair
2e5291f377 Get an LE cert for tarballs.opendev.org
Depends-On: https://review.opendev.org/663424
Change-Id: I4faa12b5d241144463ccf7ec59ef2d0b11479c35
2019-06-05 13:56:34 -07:00
Clark Boylan
e832987fca Add db backups to gitea
This isn't added as a separate role because it heavily relies on the
gitea deployment specific (docker-compose, service names, etc). If we
end up running more services with docker-compose and databases we can
probably make this reconsumable.

Change-Id: I7b9084a8a90a86f73f5b24de505978d3f286850b
2019-06-04 16:07:46 -07:00
Zuul
08c713bff0 Merge "Add opendev service-announce list" 2019-06-04 21:55:34 +00:00
Zuul
1fe34e00d4 Merge "Add control plane clouds to nodepool builder clouds.yaml" 2019-06-04 20:15:24 +00:00
Marcin Juszkiewicz
ac5acbca92 centos-mirror-update: handle non-x86 architectures
CentOS keeps non-x86 architectures in /altarch/ directory (contrary to
/centos/ one for x86-64). We have aarch64 (arm64) machines in infra and
they fail due to lack of CentOS altarch mirror.

List of wanted alternative architectures is controlled by ALTARCHS
variable (aarch64 and ppc64le enabled). As CentOS has several other
architectures too they are listed in ALTARCHS_IGNORED so we do not fetch
them.

Current CentOS mirror lands in same /mirror/centos/7/ directory. Altarch
mirrors goes to /mirror/centos/altarch/7/ one.

Change-Id: I0e110ef51c8ed301fd8280ae7fc039e3b01db92c
2019-06-03 13:07:26 +02:00
Zuul
478cd9f1e0 Merge "Fix gitea rename playbook org creation" 2019-05-31 23:37:37 +00:00
Zuul
c4be4baad2 Merge "Follow opendev renames" 2019-05-31 20:30:52 +00:00
James E. Blair
2c6e1e2061 Fix gitea rename playbook org creation
The org creation task list requires a list of the existing orgs.
Copy that from the gitea creation playbook.

Change-Id: Ia21f6211004f8dde3cacf4fb549ea8418a6d2888
2019-05-31 09:11:16 -07:00
Clark Boylan
3f1d5ccdde More repo rename playbook fixes
These fixes were either missed by the omnibus or introduced by new
changes since the big opendev migration.

Change-Id: I58e2b2c93567b47b161fdbbf143ff58738a577b8
2019-05-31 09:07:26 -07:00
Clark Boylan
48945cabc2 Serve ubuntu package content on opendev mirrors
The /var/www/mirror/ubuntu -> /afs/openstack.org/mirror/ubuntu symlink
was missing so we weren't serving ubuntu mirror content from the opendev
mirror. Add this to the list of afs content symlinks we create.

Change-Id: I10b985afbaa737033cd5c1d4dd72eb8e77f8eb32
2019-05-30 15:20:10 -07:00
Zuul
f7e14a02c9 Merge "Remove misplaced html tag" 2019-05-30 22:06:06 +00:00
Zuul
216059e134 Merge "Add opendev migration repo rename scripts" 2019-05-30 21:07:37 +00:00
Clark Boylan
7c954dba72 Add opendev service-announce list
This way we can send a single email that our users can see if subscribed
to this list instead of sending emails to all of their discuss lists.

Change-Id: I3b978a3c4e7888f14e3986628cb29a6c86bbcf61
2019-05-30 13:53:00 -07:00
Joshua Hesketh
9d51994f41 Remove misplaced html tag
Change-Id: Ifcca3ec33a4486ebcecf941c9817e92af34b84a4
2019-05-30 13:33:45 -07:00
Zuul
060f4dcbbe Merge "Project renames include keys on zuul scheduler" 2019-05-30 17:20:32 +00:00
Jeremy Stanley
4f117bcecc Project renames include keys on zuul scheduler
Add tasks to the rename_repos utility playbook for moving the
per-project secrets and ssh keys on the zuul scheduler's filesystem,
creating new namespace parent directories if they don't already
exist.

Change-Id: Iccce53953d5829bd4eb5fe4c33c9d2f195ae825c
2019-05-30 16:17:28 +00:00
Andreas Jaeger
15a5806bce Follow opendev renames
The sandbox repos moved from openstack-dev to opendev, the
zone-opendev.org and zone-zuul-ci.org as well.

Follow the rename in this repo.

Depends-On: https://review.opendev.org/657277
Change-Id: I31097568e8791cc49c623fc751bcc575268ad148
2019-05-30 16:00:30 +02:00
James E. Blair
d74c9fd62a Revert "Add tab to link from repo page to gerrit changes"
This reverts commit fe1b3cee80982fa1ec9c084196dd3b19b3f27f44.

We suspect this may be the cause of some templates going all weird:

* times are showing up as "ago%!(EXTRA string=months%!(EXTRA int64=8))"
* many strings are now showing up as lower cased (eg "explore")

Also, the link to gerrit for nova is "project:openstack/" and is
missing the "nova" portion of the name.

Change-Id: I72a06efd118ad0eae231f5ddf1a9888cb8d35aba
2019-05-29 10:10:17 -07:00
Zuul
35713be4a9 Merge "Add gitea docs" 2019-05-29 16:12:08 +00:00
Zuul
3cbd1ac93d Merge "Update opendev website to be more present tense" 2019-05-29 16:12:07 +00:00
Zuul
80a9ac2d9c Merge "Add tab to link from repo page to gerrit changes" 2019-05-29 15:51:38 +00:00
Zuul
4166abf258 Merge "Remove opendev k8s cluster from inventory" 2019-05-29 15:23:09 +00:00
Zuul
d234442dae Merge "Remove jenkins user from status.o.o" 2019-05-29 15:13:48 +00:00
Zuul
598d9106ee Merge "Remove bugday from status.o.o" 2019-05-29 15:12:00 +00:00
James E. Blair
860652fedf Add gitea docs
This is based on the recently removed cgit document, but updated
for gitea.

Change-Id: I4bb1f4b9485d15c412e4882ccc0baf031aeba87b
2019-05-29 08:06:29 -07:00
Zuul
d3d13152a7 Merge "Adding new ML for the NBMP group" 2019-05-29 14:17:48 +00:00
Jeremy Stanley
b1f95745f6 Exclude old distros from yum-puppetlabs mirror
The yum-puppetlabs mirror exceeded its 100GB quota as of April 26.
Rather than increase the quota, start excluding packages for old
platforms we don't provide like RHEL5-6 and Fedora F20-27. We could
probably get even more aggressive with it, but this get the
utilization back under 50% which is plenty of headroom for now.

Change-Id: I9665b3a2a89f991f9433fe7f45bc1bb0e0c7632b
2019-05-28 23:15:37 +00:00
Clark Boylan
b50a748d44 Switch git lb to source balance method
We were using the leastconn method which sends new connections to the
backend with the least number of connections. Unfortunately git clients
seem to have trouble with varying backend repo state (due to GC and
packing) and the thought is sending all requests from a single client to
a single backend will alleviate this.

To do this we switch to the source balance method which hashes the
source IP and finds a stable backend to talk to. This method handles
backend outages fine as it will hash to a new backend if the older one
goes offline.

Change-Id: I2c7a4ec0809a2f4ef6556833ac6a0ff3651904dd
2019-05-28 08:17:05 -07:00
Jean-Philippe Evrard
5568a461bf Exclude tmp files artifacts from syncing
It seems the openSUSE build process can leave artifacts behind,
in the form of .~tmp~ files in the mirror. I assume these are
wrongfully present.

This is a problem, as those ~tmp~ files prevent syncing the
repositories.

While it's most likely that openSUSE files will be cleaned in the
source repos, should this problem arise in the future, it's also
more robust to skip the syncing of those files.

This has the extra benefit of temporarily unblock mirroring of
openSUSE Leap 15.1 in infra, as of today.

Change-Id: I0124b992483cfda9f97960b43bddf94efa008030
2019-05-28 15:19:22 +02:00
Zuul
459bc1ab4f Merge "Start mirroring openSUSE Leap 15.1" 2019-05-27 07:33:25 +00:00
James E. Blair
8ebe74e512 Update haproxy-statsd to python3
Change-Id: I1b5c15f0c47a4bf1261abaf6d4e336aad5339c5b
2019-05-25 06:47:57 -07:00
James E. Blair
5faf89f566 Add haproxy-statsd to haproxy server
Build a container image with the haproxy-statsd script, and run that
along with the haproxy container.

Change-Id: I18be70d339df613bf9a72e115e80a6da876111e0
2019-05-24 15:40:28 -07:00