263 Commits

Author SHA1 Message Date
Zuul
4f2d4cef4c Merge "Run gerritbot with a user that will be shared with matrix-gerritbot" 2021-11-18 19:00:16 +00:00
James E. Blair
1f88e74169 Add LE config for zuul01
The service-zuul playbook isn't happy without it.

Change-Id: I46bc43b2a7daa0a17dfbb6de664df1a38f344e75
2021-11-05 14:07:49 -07:00
Clark Boylan
fd88087335 Run gerritbot with a user that will be shared with matrix-gerritbot
They have roughly the same level of access so lets align things.

Change-Id: Ifbe9dae7038345e20e8b498c87a37c519829a8cc
2021-11-05 11:24:05 -07:00
Clark Boylan
cf91bc0971 Remove the gerrit group in favor of the review group
Having two groups here was confusing. We seem to use the review group
for most ansible stuff so we prefer that one. We move contents of the
gerrit group_vars into the review group_vars and then clean up the use
of the old group vars file.

Change-Id: I7fa7467f703f5cec075e8e60472868c60ac031f7
2021-10-12 09:48:53 -07:00
Clark Boylan
63f5674e6f Switch test gerrit hostname to review99.opendev.org
Previously we had set up the test gerrit instance to use the same
hostname as production: review02.opendev.org. This causes some confusion
as we have to override settings specifically for testing like a reduced
heap size, but then also copy settings from the prod host vars as we
override the host vars entirely. Using a new hostname allows us to use a
different set of host vars with unique values reducing confusion.

Change-Id: I4b95bbe1bde29228164a66f2d3b648062423e294
2021-10-12 09:48:53 -07:00
Clark Boylan
76baae4e3f Replace testing group vars with host vars for review02
Previously we had a test specific group vars file for the review Ansible
group. This provided junk secrets to our test installations of Gerrit
then we relied on the review02.opendev.org production host vars file to
set values that are public.

Unfortunately, this meant we were using the production heapLimit value
which is far too large for our test instances leading to the occasionaly
failure:

  There is insufficient memory for the Java Runtime Environment to continue.
  Native memory allocation (mmap) failed to map 9596567552 bytes for committing reserved memory.

We cannot set the heapLimit in the group var file because the hostvar
file overrides those values. To fix this we need to replace the test
specific group var contents with a test specific host var file instead.
To avoid repeating ourselves we also create a new review.yaml group_vars
file to capture common settings between testing and prod. Note we should
look at combining this new file with the gerrit.yaml group_vars.

On the testing side of things we set the heapLimit to 6GB, we change the
serverid value to prevent any unexpected notedb confusion, and we remove
replication config.

Change-Id: Id8ec5cae967cc38acf79ecf18d3a0faac3a9c4b3
2021-10-12 09:48:45 -07:00
Jeremy Stanley
a5c1feef0f Add ptgbot serveralias for redirecting PTG site
While under development, the subdomain for the PTG site was
originally written as ptgbot.opendev.org and this is what was
communicated to event organizers. Mass communications subsequently
went out including this for URLs to the service. In order to make
the content from those announcements viable, add the additional name
to our configuration so we can redirect from it to the name we
eventually settled on.

While we're adjusting vhost metadata, make the ServerAdmin
directives between the HTTP and HTTPS vhosts for the service
consistent.

Change-Id: I726069f83b792fa31d92b759adc5c1214ca087fa
2021-10-07 19:34:16 +00:00
Ian Wienand
67c08d52ad ptgbot: rename site to ptg.opendev.org
To remain consistent with "ptg.openstack.org", we are dropping the
"bot"

Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/812757
Change-Id: I5f06ee08241a286178fcb1be5c19fee6b6086d6f
2021-10-07 09:43:39 +11:00
Ian Wienand
d48ec532d5 ptgbot: add leading # to channel name
Change-Id: I90ecb705d237e0117d0aaef1b8abee23a981ff65
2021-10-07 09:09:52 +11:00
Jeremy Stanley
ad16067ae8 Finish ptgbot configuration
Set the channel we want ptgbot joining in production with a group
var, like we do for statusbot's channel list. Correct the password
var name to match what's used in the template for production (and
matches the override set in our private hostvars on the bastion).
Clean up the unnecessary auth nicks list which was copied from the
statusbot config but is entirely unused. Also get rid of some
unnecessary empty lines in the defaults as they really don't make
the file any more readable.

Change-Id: Id026b89d642eae13feba374e4f3ec610b543e530
2021-10-06 19:06:39 +00:00
Kendall Nelson
a24310c17f Setup Letsencrypt for ptgbot site
Depends-on: https://review.opendev.org/c/opendev/zone-opendev.org/+/804790

Change-Id: I8812628a3b021180e7c5d33ccd0cd5c2f0cc4745
2021-10-06 15:39:25 +11:00
Clark Boylan
f1bcb6a586 Set a gerrit replication timeout of 15 minutes
We are seeing that replication tasks occasionally sit around forever and
have had to take manual intervention. One theory is that this is related
to networking between the gerrit server and the gitea servers. We don't
set maxRetries which means replication should be retried infinitely
which means if we hit the timeout we should try again. 15 minutes was
sort of arbitrarily chosen as ~twice the time it takes to clone a large
repo like nova.

Change-Id: Iec2536ad149a2e625a1f0107b9fcee3079493607
2021-09-15 16:25:04 -07:00
Clark Boylan
6c115cf29e Add support for Ubuntu Focal to our mailman ansible
This switch testing of lists.openstack.org to Focal and we make a CGI
env var update to accomodate newer mailman.

Specifically newer mailman's CGI scripts filter env vars that it will
pass through. We were setting MAILMAN_SITE_DIR to vhost our mailman
installs with apache2, but that doesn't pass the filter and is removed.
HOST is passed through so we update our scripts, apache vhost configs,
exim, and init scripts to use the HOST env var instead.

Change-Id: I5c8c70c219669e37b7b75a61001a2b7f7bb0bb6c
2021-09-13 09:10:00 -07:00
Zuul
da558e10c3 Merge "Introduce iweb cloud configs" 2021-08-31 23:38:46 +00:00
Mathieu Gagné
d518f8c02c Introduce iweb cloud configs
INAP mtl01 region is now owned by iWeb. This updates the cloud launcher
to use the new name and instructs the mirror in this cloud to provision
ssl certs for the old inap and new iweb names as well as updating
clouds.yaml files.

Change-Id: I1256a2e24df1c79dea06716ae4dfbcfe119c13f8
2021-08-31 13:42:22 -07:00
Jeremy Stanley
2fbf6d9e7a Stop managing OpenStackID servers
The Open Infrastructure Foundation's developers who maintain the
OpenStackID software are taking over management of the site itself,
and have deployed it on new servers. DNS records have already been
updated to the new IP address, so it's time to clean up our end in
preparation for deleting the old servers we've been running.

OpenStackID is still used by some services we run, like RefStack and
Zanata, and we're still hosting the OpenStackID Git repository and
documentation, so this does not get rid of all references to it.

Change-Id: I1d625d5204f1e9e3a85ba9605465f6ebb9433021
2021-08-31 19:53:13 +00:00
Andreas Jaeger
afb343f4bf Retire openstack-i18n-de mailing list
The mailing list is dead, the last post was about the current owner
leaving:
http://lists.openstack.org/pipermail/openstack-i18n-de/2020-September/000165.html

Let's retire it.

Change-Id: Iad93a65e8f68397ba3f83d8114c30ae78a930e24
2021-08-23 15:54:10 +02:00
Zuul
668aa77c9b Merge "Move #zuul from OFTC to Matrix" 2021-08-21 14:57:09 +00:00
James E. Blair
ac1dd4eedd Assume gitea reverse proxy
We now depend on the reverse proxy not only for abuse mitigation but
also for serving .well-known files with specific CORS headers.  To
reduce complexity and avoid traps in the future, make it non-optional.

Change-Id: I54760cb0907483eee6dd9707bfda88b205fa0fed
2021-08-20 22:06:03 -07:00
James E. Blair
cdbfe6b97e Move #zuul from OFTC to Matrix
Zuul is moving to an unbridged Matrix room.  Remove eavesdrop from
the OFTC room, and add the Matrix room to the two new Matrix bots.

Change-Id: I9bf34c1f67c6dac41c3761f8ccde4d7fa76bbf89
2021-08-20 14:44:44 -07:00
James E. Blair
fd4fd57409 Remove port 22 from webservers extra ports
This isn't necessary since it's hard-coded into the file.  Let's
not add it where it isn't needed lest we confuse ourselves into
thinking it's necessary.

Change-Id: I011c647bb85e145e55fb6feb19facdedec180bf1
2021-08-11 14:21:34 -07:00
James E. Blair
8d76a7cd99 Test port 9001 on eavesdrop
We merged change I9459e47ecfd19b27b7adcaee9ce91f80d51c124d which
should have opened this port but did not.  Add testing for it.

Remove eavesdrop from webservers group

This was overridding the custom iptables ports that were being set
in the eavesdrop group vars file.  There appears to be no other use
for the webservers group.

Change-Id: I7109f1472176ff39482f9bdfc8462e5f525f791c
2021-08-11 14:20:41 -07:00
Zuul
92ead4baa1 Merge "Remove the mysql support from our gerrit role and image" 2021-08-10 23:32:37 +00:00
Clark Boylan
75e0cf106a Remove the mysql support from our gerrit role and image
We are now using the mariadb jdbc connector in production and no longer
need to include the mysql legacy connector in our images. We also don't
need support for h2 or mysql as testing and prod are all using the
mariadb connector and local database.

Note this is a separate change to ensure everything is happy with the
mariadb connector before we remove the fallback mysql connector from our
images.

Change-Id: I982d3c3c026a5351bff567ce7fbb32798718ec1b
2021-08-10 13:06:54 -07:00
Tristan Cacqueray
32a38a4b83 Add gerritbot-matrix health check and expose prometheus monitoring
This change enables monitoring the gerritbot-matrix service metrics.

Change-Id: I9459e47ecfd19b27b7adcaee9ce91f80d51c124d
2021-08-08 17:35:45 +00:00
Zuul
84091f5de4 Merge "Improve gerrit known_hosts management" 2021-08-06 17:10:19 +00:00
Zuul
85e32638db Merge "Serve gating.dev static website" 2021-08-06 00:20:25 +00:00
Zuul
47d1737cd7 Merge "Add mailing list for FLOSS MOOC" 2021-08-02 17:20:31 +00:00
Clark Boylan
f6a0bf7be5 Improve gerrit known_hosts management
Previously we were only managing root's known_hosts via ansible but even
then this wasn't happening because the gerrit_self_hostkey var wasn't
set anywhere. On top of that we need to manage multiple known_hosts
because gerrit must recognize itself and all of the gitea servers.
Update the code to take a dict of host key values and add each entry to
known_hosts for both the root and gerrit2 user.

We remove keyscans from tests to ensure that this update is actually
working.

Change-Id: If64c34322f64c1fb63bf2ebdcc04355fff6ebba2
2021-08-02 09:53:27 -07:00
Kendall Nelson
fc87c924a3 Add mailing list for FLOSS MOOC
Currently being collaboratively developed here:

https: //gitlab.com/mooc-floss/mooc-floss
Change-Id: I2002ee48878e816544b08012668c66337ab4707a
2021-08-02 16:17:40 +00:00
James E. Blair
af33336ca9 Serve gating.dev static website
Change-Id: I5219656f770842c8b222b6685ae1f0d7126b8065
2021-07-29 17:15:38 -07:00
Ian Wienand
c1278d18bb Remove review-test
With our system-config-run gerrit/review jobs we have much less need
for a dedicated server to stage changes on.  Remove in prepartion of
server cleanup.

Change-Id: I9430f7a2432324a184e3a4f7e41f9e5150c0200c
2021-07-21 13:12:43 +10:00
Ian Wienand
e79e3a2f04 Remove review01 references
This server is no longer in production, so remove the various
references to it.

Change-Id: I2cdd8052c48713e9ba648be20ccad5069d5fe40e
2021-07-20 11:57:10 +10:00
Zuul
cf7c95acb1 Merge "review02: skip ~gerrit2/tmp in backup" 2021-07-19 08:14:56 +00:00
Ian Wienand
ede3a6390c review02: skip ~gerrit2/tmp in backup
Change-Id: Ifcd8f6195b70592288f9a82fea170ae0def4d57a
2021-07-19 17:05:13 +10:00
Ian Wienand
0142bc10eb backups: add review02.opendev.org
Start backing up the new review server.  Stop backing up the old
server.  Fix the group matching test for the new server.

Change-Id: I8d84b80099d5c4ff7630aca9df312eb388665b86
2021-07-19 15:29:42 +10:00
Zuul
f1b559bb7a Merge "review02: move out of staging group" 2021-07-19 04:49:37 +00:00
Ian Wienand
8607ff7d81 review02: move out of staging group
This moves review02 out of the review-staging group and into the main
review group.  At this point, review01.openstack.org is inactive so we
can remove all references to openstack.org from the groups.  We update
the system-config job to run against a focal production server, and
remove the unneeded rsync setup used to move data.

This additionally enables replication; this should be a no-op when
applied as part of the transition process is to manually apply this,
so that DNS setup can pull zone changes from opendev.org.

It also switches to the mysql connector, as noted inline we found some
issues with mariadb.

Note backups follow in a separate step to avoid doing too much at
once, hence dropping the backup group from the testing list.

Change-Id: I7ee3e3051ea8f3237fd5f6bf1dcc3e5996c16d10
2021-07-18 19:45:35 -07:00
Zuul
b895af4d35 Merge "Remove paste01.openstack.org" 2021-07-16 03:03:50 +00:00
Ian Wienand
5e52befdfa Remove paste01.openstack.org
This has been replaced by paste01.opendev.org and Ansible deployment.

Change-Id: I0f8f5374a3f5d269b317bde4ae2b37435e0871d5
2021-07-15 23:25:10 +00:00
Ian Wienand
d4c613a07a Add paste01.opendev.org to backup
Change-Id: Iec6b916bd27a5333d28d1fdc931d4f41165bf50c
2021-07-15 15:02:52 +10:00
Clark Boylan
25d2fdcc3f Add warning to inventory about zuul gerrit server config
Let's avoid changing this and breaking Depends-On again by adding an
explicit warning to the code that sets the config.

Change-Id: Idcb77d8b0b53c56ea7f15f18e001f8bc9a001c98
2021-07-13 10:32:45 -07:00
Clark Boylan
2c06a86915 Talk to review.o.o instead of review01.o.o
Talking to review01.o.o in the Zuul gerrit connection config broke
depends-on handling as the urls would all need to be
https://review01.opendev.org/123456 and then later
https://review02.opendev.org/123456 but people use
https://review.opendev.org/123456.

This change was made to simplify DNS updates during the gerrit server
move but we should be able to handle those via manual landing of changes
and running of playbooks instead. Partially revert
e05257e1b7b70b18cb7b1349278e2c786a565512 to fix the depends-on handling.

Change-Id: Ie628b2627c263d88e466205af2a3d0418d6df7d3
2021-07-13 10:27:36 -07:00
Zuul
51480ca77e Merge "Add paste service" 2021-07-13 00:07:03 +00:00
Zuul
f45f5f9626 Merge "Connect Zuul to review01.opendev.org" 2021-07-12 00:11:27 +00:00
Ian Wienand
916c1d3dc8 Add paste service
The paste service needs an upgrade; since others have created a
lodgeit container it seems worth us keeping the service going if only
to maintain the historical corpus of pastes.

This adds the ansible to deploy lodgeit and a sibling mariadb
container.  I have imported a dump of the old data as a test.  The
dump is ~4gb and imported it takes up about double that; certainly
nothing we need to be too concerned over.  The server will be more
than capable of running the db container alongside the lodgeit
instance.

This should have no effect on production until we decide to switch
DNS.

Change-Id: I284864217aa49d664ddc3ebdc800383b2d7e00e3
2021-07-07 15:12:04 +10:00
Zuul
635b7dd682 Merge "Add gating.dev zone to ADNS" 2021-06-28 22:11:06 +00:00
James E. Blair
066c2ec4e1 Add gating.dev zone to ADNS
Depends-On: https://review.opendev.org/798374
Change-Id: I901d79c1fceec5566dfd4917b2c7903ffc443acf
2021-06-28 19:39:41 +00:00
Ian Wienand
e05257e1b7 Connect Zuul to review01.opendev.org
Point the Zuul scheduler at review01.opendev.org instead of the CNAME
review.opendev.org.  This avoids chicken-egg issues because Zuul
actually updates the DNS entries.

Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/798242
Change-Id: I1f6054fdab0fe0fcb311686d6af6454b6a714666
2021-06-28 14:36:08 +10:00
Ian Wienand
0e9b950086 Add eavesdrop01.opendev.org to backup group
This saves a copy of our channel/meeting logs.

Change-Id: I376d1426573416ff0c2e633fa40e4d93adc89483
2021-06-23 10:48:38 +10:00