The status.openstack.org server is offline now that it no longer
hosts any working services. Remove all configuration for it in
preparation for retiring related Git repositories.
Also roll some related cleanup into this for the already retired
puppet-kibana module.
Change-Id: I3cfcc129983e3641dfbe55d5ecc208c554e97de4
We indicated to the OpenStack TC that this service would be going away
after the Yoga cycle if no one stepped up to start maintaining it. That
help didn't arrive in the form of OpenDev assistance (there is effort
to use OpenSearch external to OpenDev) and Yoga has released. This means
we are now clear to retire and shutdown this service.
This change attempts to remove our configuration management for these
services so that we can shutdown the servers afterwards. It was a good
run. Sad to see it go but it wasn't sustainable anymore.
Note a follow-up will clean up elastic-recheck which runs on the status
server.
Depends-On: https://review.opendev.org/c/opendev/base-jobs/+/837619
Change-Id: I5f7f73affe7b97c74680d182e68eb4bfebbe23e1
We already cleaned most of this up[1] but we need to clean up one more
unused file.
[1] https://review.openstack.org/#/c/452558/
Change-Id: I0fa10b447bd42047f9e18336549ad1bdd5eab52b
We have shifted over to using ansible for managing the listservs.
This also updates our service docs to point at the corret ansible and
not puppet.
Change-Id: I76f01ff1479c5af0a502a060aac2baa1ab622b21
This migrated to Ansible with
Idbe084f13f3684021e8efd9ac69b63fe31484606. Remove the now unused
puppet components.
Change-Id: I500d6eefcb64f4941e216b8590f4cd60ceec0811
The Limesurvey service hosted at survey.openstack.org was a beta
which saw limited use. The platform it runs on, Xenial, is now EOL
from Ubuntu/Canonical and in order to upgrade to a newer
distribution release we would need to rewrite all the configuration
management (the version of Puppet supported by newer Ubuntu is not
backward-compatible with what we've been running).
If a similar service becomes interesting to users of our
collaboratory in the future, it will need to be reintroduced with
freshly written configuration management anyway. The old configs and
documentation remain in our Git history should anyone wish to use
them as inspiration.
Change-Id: I59b419cf112d32f20084ab93eb6f2417a7f93fdb
Once we are satisfied that we have disabled the inputs to firehose we
can land this change to stop managing it in config management. Once that
is complete the server can be removed.
Change-Id: I7ebd54f566f8d6f940a921b38139b54a9c4569d8
The OpenEdge cloud has been offline for five months, initially
disabled in I4e46c782a63279d9c18ff4ba2944c15b3027114b, so go ahead
and clean up lingering references. If it is restored later, this can
be reverted fairly easily.
Depends-On: https://review.opendev.org/783989
Depends-On: https://review.opendev.org/783990
Change-Id: I544895003344bc8202363993b52f978e1c07d061
It looks like we missed these in cleanups for the old puppet-managed
mirror-update server (I5f82139c981c2716f568b15b118690e943b02d52).
These are unused.
Change-Id: Ia79920a7567d73d311f37d73e10c1396d09ddf93
With our increased ability to test in the gate, there's not much use
for review-dev any more. Remove references.
Change-Id: I97e9865e0b655cd157acf9ffa7d067b150e6fc72
Our Mailman site templates and similar content contain links to an
old openstack-security page on the foundation-run site which no
longer exists. Correct this to the OpenStack community's security
site, which should be much more stable.
Change-Id: I9577540319c53f76afc40a33b2c5697280397149
The hound project has undergone a small re-birth and moved to
https://github.com/hound-search/hound
which has broken our deployment. We've talked about leaving
codesearch up to gitea, but it's not quite there yet. There seems to
be no point working on the puppet now.
This builds a container than runs houndd. It's an opendev specific
container; the config is pulled from project-config directly.
There's some custom scripts that drive things. Some points for
reviewers:
- update-hound-config.sh uses "create-hound-config" (which is in
jeepyb for historical reasons) to generate the config file. It
grabs the latest projects.yaml from project-config and exits with a
return code to indicate if things changed.
- when the container starts, it runs update-hound-config.sh to
populate the initial config. There is a testing environment flag
and small config so it doesn't have to clone the entire opendev for
functional testing.
- it runs under supervisord so we can restart the daemon when
projects are updated. Unlike earlier versions that didn't start
listening till indexing was done, this version now puts up a "Hound
is not ready yet" message when while it is working; so we can drop
all the magic we were doing to probe if hound is listening via
netstat and making Apache redirect to a status page.
- resync-hound.sh is run from an external cron job daily, and does
this update and restart check. Since it only reloads if changes
are made, this should be relatively rare anyway.
- There is a PR to monitor the config file
(https://github.com/hound-search/hound/pull/357) which would mean
the restart is unnecessary. This would be good in the near and we
could remove the cron job.
- playbooks/roles/codesearch is unexciting and deploys the container,
certificates and an apache proxy back to localhost:6080 where hound
is listening.
I've combined removal of the old puppet bits here as the "-codesearch"
namespace was already being used.
Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
The active releases according to [1] are octopus and nautlius. Remove
the old releases from our mirroring. This needs manual cleanup of the
jobs and volumes -- I will do this manually as this is mostly about
clearing out old things before moving the mirroring to Ansible.
[1] https://docs.ceph.com/en/latest/releases/
Change-Id: I050f737521fa6837f3b6b52b8028a839a29f7bd2
In order to collect historical statistics on usage of our mailing
lists, we need an index of not only the current lists (which we
could get from Mailman) but also retired lists (which could only be
found by knowing the URL to their archives). Ultimately we should
publish hyperlinks to these so they'll continue to be indexed by
search engines, but for now start with structured YAML, which we
could later use to build that too.
Because the only way to determine the names of retired lists is from
the listserv's filesystem, we'll run a simple script once daily to
refresh the index and keep it in the Web root alongside the
robots.txt file. In the future, this could be triggered instead by
addition of new mailing lists, though while we're still managing
them with Puppet it's not clear how to go about doing that.
Of course restrict this to only indexing public list archives, as
privately-archived lists won't be accessible to the general public
by design.
Change-Id: Ibe3175a56831b7a43698d6fe454d70e93fcd0bc7
This is to replace the puppet managed openstack.org server
Change-Id: I0e3586befd922cb56d1a0ec9c9cb650add9b225d
Depends-On: https://review.opendev.org/728314
These are to replace the puppet-based openstack.org mirrors
Depends-On: https://review.opendev.org/728308
Change-Id: Ibdce99daa514fb445f1f8389e7c052ee151057ea
People are starting to use this service so having performance metrics
over time is a good thing. We also want to avoid having our cert expire
unexpectedly.
Change-Id: I744b3e68f8f483b36c0d8ecb6f6f46a484a3577a
New opendev.org CI mirrors for OVH regions. The old BHS1 mirror was
in the openstack.org domain, so is added new. There was an old GRA1
mirror in the opendev.org domain, so remote it and increment the
ordinal in its short hostname to avoid a collision in the inventory
cache.
This is being done to switch to un-billed flavors in this provider,
to simplify internal billing for their donation of resources.
Change-Id: I05770856b5704aa438ed6bc54ec42ba9efb5cd2a
This sets up a robots.txt on our lists servers. To start this file
prevents SEMrush bot from indexing our lists as that has been causing
lists.openstack.org to OOM with many listinfo processes started by
Apache.
We've avoided this OOM by manually configuring this robots.txt. Other
things we have ruled out are bup and input email causes qrunner's to
grow unexpectedly large. Fairly confident this bot is the trigger.
Note this fixes testing by adding 'hieradata' to set listpassword var.
Depends-On: https://review.opendev.org/724389
Change-Id: Id4f6739a8cf6a01f9796fa54c86ba1af3e31fecf
After the big OpenDev rename, these repos got renamed again. Update the
redirects for git.airshipit.org and git.starlingx.io to point to the
current location.
Update test_static.py for this, change the test repo since
airship-in-a-bottle was first renamed to in-a-bottle and later to
airship-in-a-bottle.
Change-Id: I71b786cd528aac9ae68464618db02e22cd4c0b5b
zuul and nodepool now life in opendev, avoid double redirects and
redirect directly to final location.
Change-Id: Ia55d76b24f07ec64cb55055955c4549f3706a95b
This adds the Open Edge (formerly Fortnebula) CI mirror.
Change-Id: I1ccf2a602f8a41e00bc64a9516a326cc07d9b254
Depends-On: https://review.opendev.org/711787
Sister change for Ia5caff34d3fafaffc459e7572a4eef6bd94422ea and
removing earlier references to the mirror server in preparation for
building and adding the new one.
Change-Id: I7d506be85326835d5e77a0c9c461f2d457b1dfd3
This reverts commit c25e91f49632d8e187f35807f250567446bd5102.
This script parses the Apache logs and writes out a local count of the
404 data to files.openstack.org, and then exports it via
files.openstack.org.
As part of the spec [1] we're trying to remove publishing from local
volumes, in general.
Since this is not widely used, there is only one link to it, it's not
discoverable from the landing page of files.openstack.org (which just
shows the afs directory listing), it has a very long latency making it
not that useful for debugging and grepping the logs there have been no
accesses in the past 2 weeks (as far back as logs go) I propose we
remove it.
If we want to retain this, we should publish the output alongside the
docs AFS volume. That could certainly be done by distributing the
docs keytab to the host and having it write out in a similar cron job.
Another option could be to setup a keypair for remote login and keep
that as a secret in Zuul, and do the same from a periodic job
(complicated by apache logs being root only, so needs some sudo magic
or similar). Or, we could figure out an altogether better, privacy
respecting client analytics solution.
[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/retire-static.html
Depends-On: https://review.opendev.org/709036
Change-Id: Iccf24a72cf82592bae8c699f9f857aa54fc74f10
A bunch of the sites we've started managing LE certs for are not
getting their expirations checked, so fix that. In particular, sites
recently moved off the multi-domain SAN cert for the old
static.openstack.org server (omitted the logs site as it's
deprecated), and many of the rebuilt CI mirrors (with the exception
of mirror01.gra1.ovh.opendev.org which is presently in a SHUTOFF
state for unknown reasons). Also add graphite which was previously
missed, and review-dev because we can now that it's no longer
sporting snakeoil.
When this merges, we're also going to start getting alerts for an
expired cert on mirror.gra1.ovh.opendev.org, unless someone gets a
chance to look into it first.
Change-Id: I98a98e0d2ff081c51c33d980274f3ee8c0266802
The insecure-ci-registry.opendev.org service uses an X.509 cert on
5000/tcp, so we should track this to catch when it's going to
expire.
Change-Id: I5d18599e5b5b258ce158f964cb1ff95df6dc6d92
The ssldomains file we use for our cert check is getting longish,
and sorting it will make entries easier to find.
Change-Id: Iad182ecee45274d6c8f336a97d20a3130e4b8abe
Now that opendev.org backends requests certs unique to each backend we
should check these backends directly and not only through the frontend.
This was if a specific backend doesn't end up updating with LE properly
we will catch it.
Change-Id: Icabb1bcb725937da45ae9aaef2c9da412a30a319
This runs gerrit in a container on review-dev01 using podman.
Remove an unused web_server.py file that we found from copying it
from puppet to ansible.
Change-Id: I399d3cf8471bc8063022b0db0ff81718b2ee2941
Ceph Nautilus is released and the official mirror
is available. This adds the Ceph Nautilus mirror
so we can sync it for Stretch and Bionic.
Based on the same change that was done when Mimic
was released [1]
[1] https://review.opendev.org/#/c/571989/
Change-Id: I9424d1f4df58acde8ea70dc16283d4de89189bae
