10534 Commits

Author SHA1 Message Date
Zuul
e04a13b3ba Merge "Correct path in mk-archives-index cronjob on lists" 2021-02-16 01:16:03 +00:00
Thierry Carrez
75e15d765d PTGBot is now openinfraptg on #openinfra-events
Channel for PTG discussion was moved to #openinfra-events, and
the bot was renames "openinfraptg" to match the extended scope
of the event.

Change-Id: I52718358ddb4a199d24fc6da6e71f81c646da1f2
2021-02-10 14:33:55 +01:00
Ian Wienand
01990670c9 translate: backup zanata db directly to borg
As noted inline, a recent mysql client update has broken the
"--all-databases" flag, at least for the client version and very old
server version we use.

Emperically, dumping individual databases still works with this
client.  Switch this to stream the db directly into borg.

Ignore the old backups and remove the bup backup while we are here,
since this is all borg now.

Change-Id: I5fe762a003ce2c2ba4830367be87598f67f7e763
2021-02-05 14:05:24 +11:00
Ian Wienand
f9184ce323 ask: stream db backup
Despite be deprecated, the ask server is our 3rd biggest backup.  Even
though the site is R/O we're still backing up the fresh rotations of
the gzipped backups every day.

To reduce the incremental space requirements, move to our plain-text
streaming for the db backup.  This just needs a file dropped in /etc;
see the backup-borg role README documentation.  We do this in puppet
to avoid complexity adding this deprecated service to ansible.  This
then excludes the on-disk db backup dir.

Drop the bup backups while we are here.

Change-Id: Icfd81aca58b9a0dc3a3b74de04c1b00f03160327
2021-02-05 13:24:57 +11:00
Zuul
e3e555d9fd Merge "Remove AFS puppet" 2021-02-04 03:52:44 +00:00
Radosław Piliszek
34ee1a56d9 ask.o.o: Fix two issues with ro message
1) The string is interpolated into JavaScript string which is
   delimited using double quotation marks - using double quotation
   marks in it breaks JavaScript parsing. The impact is unknown
   but at least some JavaScript code does not get executed later.

2) The anchor was unproperly closed causing void anchor to appear.
   This is clearly visible on the rendered page.

Change-Id: I90cdcdd81c6af67f940c1811b1b9c05f9309ba15
2021-01-28 20:33:18 +01:00
smarcet
1ff4ccadaa OpenstackId v3.0.18
* updated user rest api for CRUD
* improved session managing ( iframe)
* fixed admin UI issues
* fixed CORS issues ( upgraded middleware)
* bug fixing

Change-Id: I1344b5a5b0df8c24ec4e13ab2aae0dcf5339b6ae
Signed-off-by: smarcet <smarcet@gmail.com>
2021-01-26 17:16:27 -03:00
smarcet
fcdb1762d6 OpenstackId v3.0.17
* updated user rest api for CRUD
* improved session managing ( iframe)
* fixed admin UI issues
* fixed CORS issues ( upgraded middleware)
* bug fixing

Change-Id: I0ed6092f62eb5ee6e80cbaabc03d79633dee9aaa
Signed-off-by: smarcet <smarcet@gmail.com>
2021-01-25 11:38:35 -03:00
Ian Wienand
61e9d0948a Remove AFS puppet
This has all been replaced by Ansible roles and is no longer used

Change-Id: Ic807498ad3ca4f305b168464b86fe197a61b4d13
2021-01-21 07:08:37 +11:00
Jeremy Stanley
25697d62bb Correct path in mk-archives-index cronjob on lists
The mk-archives-index command is installed in /usr/local/sbin, so
add that to the path of the cronjob which calls it. Otherwise,
http://lists.opendev.org/archives.yaml is empty and engagement
statistics cannot be generated.

Change-Id: Ib49e8a7b78f8cb9cb385ba09b39e3f940cd17ad6
2021-01-16 15:27:02 +00:00
Zuul
c818a82376 Merge "Publish structured data listing our ML archives" 2021-01-14 03:38:55 +00:00
Ian Wienand
368466730c Migrate codesearch site to container
The hound project has undergone a small re-birth and moved to

 https://github.com/hound-search/hound

which has broken our deployment.  We've talked about leaving
codesearch up to gitea, but it's not quite there yet.  There seems to
be no point working on the puppet now.

This builds a container than runs houndd.  It's an opendev specific
container; the config is pulled from project-config directly.

There's some custom scripts that drive things.  Some points for
reviewers:

 - update-hound-config.sh uses "create-hound-config" (which is in
   jeepyb for historical reasons) to generate the config file.  It
   grabs the latest projects.yaml from project-config and exits with a
   return code to indicate if things changed.

 - when the container starts, it runs update-hound-config.sh to
   populate the initial config.  There is a testing environment flag
   and small config so it doesn't have to clone the entire opendev for
   functional testing.

 - it runs under supervisord so we can restart the daemon when
   projects are updated.  Unlike earlier versions that didn't start
   listening till indexing was done, this version now puts up a "Hound
   is not ready yet" message when while it is working; so we can drop
   all the magic we were doing to probe if hound is listening via
   netstat and making Apache redirect to a status page.

 - resync-hound.sh is run from an external cron job daily, and does
   this update and restart check.  Since it only reloads if changes
   are made, this should be relatively rare anyway.

 - There is a PR to monitor the config file
   (https://github.com/hound-search/hound/pull/357) which would mean
   the restart is unnecessary.  This would be good in the near and we
   could remove the cron job.

 - playbooks/roles/codesearch is unexciting and deploys the container,
   certificates and an apache proxy back to localhost:6080 where hound
   is listening.

I've combined removal of the old puppet bits here as the "-codesearch"
namespace was already being used.

Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
2020-11-20 07:41:12 +11:00
Zuul
d3a53e8ec0 Merge "Remove mirror-update server and related puppet" 2020-11-09 21:07:11 +00:00
Zuul
00c496e879 Merge "Add service-incident@opendev mailing list" 2020-10-30 15:37:51 +00:00
Zuul
b72845c274 Merge "Cleanup grafana.openstack.org" 2020-10-29 05:15:00 +00:00
Ian Wienand
c49ece9204 Cleanup grafana.openstack.org
The opendev.org server is in production, cleanup the old puppet-based
host.

Change-Id: I6db3ce929226a23b96234b52ece8b17f4c6a326a
2020-10-29 07:59:42 +11:00
Ian Wienand
f8852b76fb Remove mirror-update server and related puppet
This has all transitioned to Ansible and the mirror-update.opendev.org
server now.

Change-Id: I5f82139c981c2716f568b15b118690e943b02d52
2020-10-28 11:39:54 +11:00
Ian Wienand
10b2cd5fed reprepo: enable cron jobs
Enable the Ansible based cron jobs, and disable the puppet host
versions to cut over the mirroring to the new server.

Change-Id: I0ffb1c484e64e67f5a5017dc3c3c8ebcdc3845c8
2020-10-28 11:29:26 +11:00
Zuul
89a1edce3d Merge "Remove old debian-ceph mirrors" 2020-10-27 02:57:44 +00:00
Jeremy Stanley
abc66ed38c Add service-incident@opendev mailing list
Create a mailing list for private coordination of security incidents
for the OpenDev Collaboratory. The intent is that this can be used
to share sensitive information between sysadmins and council members
in the event of any suspected breach. For the sake of transparency,
all information discussed on this list which can safely be made
public should also be communicated to the service-announce or
service-discuss mailing lists at the earliest opportunity.

Change-Id: I32bef68eb7019261471c167d19eee733457078a2
2020-10-22 16:16:04 +00:00
Clark Boylan
fa362b813c More old apache acl cleanups
We can rely on Require instead of Order, Allow, Deny, Satisfy since we
are all on apache 2.4 now. This simplifies reasoning about acl rules.

Change-Id: Idedba1558ccaa1c753d1175e356bf26a8d4b1084
2020-10-16 11:16:26 -07:00
Ian Wienand
961bab63d9 Remove old debian-ceph mirrors
The active releases according to [1] are octopus and nautlius.  Remove
the old releases from our mirroring.  This needs manual cleanup of the
jobs and volumes -- I will do this manually as this is mostly about
clearing out old things before moving the mirroring to Ansible.

[1] https://docs.ceph.com/en/latest/releases/

Change-Id: I050f737521fa6837f3b6b52b8028a839a29f7bd2
2020-10-16 14:16:19 +11:00
smarcet
807ea2608e OpenstackId v3.0.16
added new search criteria for endpoint
GET /api/v1/users

primary_email (==,@=)

Change-Id: Ib643a8c1ba4e79444463777197fc86a64a1912be
Signed-off-by: smarcet <smarcet@gmail.com>
2020-10-15 00:23:08 -03:00
smarcet
e394198d03 OpenstackID v3.0.15
* updated registration emails
* updated registration form
* updated password policies
* bug fixing

Change-Id: Ibd644e9daa9bd345cf883db3dfa75b58b4ad7a18
Signed-off-by: smarcet <smarcet@gmail.com>
2020-10-12 17:20:54 -03:00
smarcet
2f970563c0 OpenstackId config updates
Added cloud storage config

Change-Id: I39cefce0c1910df0fc051817193e14e5a38c3a1e
Signed-off-by: smarcet <smarcet@gmail.com>
2020-09-21 17:40:19 -03:00
smarcet
8e1d69a674 OpenStackID v3.0.14
Change-Id: I3a7fa4fbfb16bd981f8e80fccb774db6a4f0649a
Signed-off-by: smarcet <smarcet@gmail.com>
2020-09-21 16:11:01 -03:00
smarcet
d7a418c024 Updated openstack id to include
message broker configuration

Change-Id: Ia3fe6ddbe92b354b81f5572ba3f6fba60ac3ce31
Signed-off-by: smarcet <smarcet@gmail.com>
2020-09-21 09:02:09 -03:00
Jeremy Stanley
c147dc4b5b Publish structured data listing our ML archives
In order to collect historical statistics on usage of our mailing
lists, we need an index of not only the current lists (which we
could get from Mailman) but also retired lists (which could only be
found by knowing the URL to their archives). Ultimately we should
publish hyperlinks to these so they'll continue to be indexed by
search engines, but for now start with structured YAML, which we
could later use to build that too.

Because the only way to determine the names of retired lists is from
the listserv's filesystem, we'll run a simple script once daily to
refresh the index and keep it in the Web root alongside the
robots.txt file. In the future, this could be triggered instead by
addition of new mailing lists, though while we're still managing
them with Puppet it's not clear how to go about doing that.

Of course restrict this to only indexing public list archives, as
privately-archived lists won't be accessible to the general public
by design.

Change-Id: Ibe3175a56831b7a43698d6fe454d70e93fcd0bc7
2020-09-11 00:47:22 +00:00
Zuul
8c599a5bd0 Merge "Add ceph octopus mirrors" 2020-09-10 22:32:05 +00:00
Mohammed Naser
322afab352 Add ceph octopus mirrors
Change-Id: I8876b89088bf1530c99edd08f644efe03d2cf867
2020-09-10 21:31:16 +00:00
Zuul
d30861adb5 Merge "Add zuul-jobs-failures list" 2020-09-09 14:34:26 +00:00
Zuul
d9e1e64497 Merge "Improved ask read-only message" 2020-09-02 13:26:56 +00:00
Zuul
f282b69801 Merge "Mirror Puppetlabs puppet for Ubuntu Focal" 2020-09-02 00:12:46 +00:00
Sorin Sbarnea
19457efb3c Add zuul-jobs-failures list
Add list to be used to monitor failures of periodic checking jobs.

Change-Id: I0df2bf01d7ddf290326f1d83cdb73bcc91cf81f7
2020-08-28 15:59:54 +01:00
Sorin Sbarnea
ef5645baa7 Improved ask read-only message
Makes the read-only message more detailed and includes hyperlinks.

Change-Id: I01f76949276962971246ce760c371e5bd010cb02
2020-08-24 08:24:07 +01:00
Thierry Carrez
06f725519d Make ask.openstack.org read-only
Nobody maintains our askbot website, and questions there go
unanswered. In the spirit of simplification, make the site
read-only (so that old answers can still be found) and redirect
users to the openstack-discuss mailing-list and Stack Overflow
(which has a decent openstack community answering questions).

Read-only config values documented at:
https://github.com/ASKBOT/askbot-devel/blob/master/askbot/conf/access_control.py

Change-Id: I33d9d7c87a5a17138fcdc37ee8f8b16cda2248d5
2020-08-17 15:07:21 +02:00
Ian Wienand
3eabc630c4 Fix debian-docker mirroring
Unfortunately we can't mix the distributions here, because upstream
keeps the same filename, built differently, in each distributions
separate pool.  So we can't combine it back into one pool.

Mirror each into a separate subdirectory.

Change-Id: I728d38daf9a953a64364689da0648c9339a27693
2020-08-14 13:20:43 +10:00
Ian Wienand
69a92d0d9d Add arm64 to debian-docker mirroring
Change-Id: Ice04f98131a9a6ab1eb733c93bce713748b84f56
2020-08-14 10:45:55 +10:00
Tobias Urdin
8dc369da97 Mirror Puppetlabs puppet for Ubuntu Focal
Change-Id: Id054803b1641945f900b89bb971044ca020d7194
2020-07-15 20:25:03 +02:00
Marcin Juszkiewicz
5e79b51695 UCA: mirror Victoria packages for Focal
Change-Id: Ie2195c8d0d9b3832a14a8bc7a1d761eb4aa53ee5
2020-07-02 16:11:26 +02:00
Clark Boylan
6f986a4fcc Remove elasticsearch01
This host hasn't existed in years. Make that clearer in system-config by
removing it.

Change-Id: Ie11151c00ab32ea249be0f539952660cefc0fa2d
2020-06-17 14:51:58 -07:00
Zuul
45b04b5074 Merge "Cleanup old puppet management of release-volumes.py" 2020-06-15 21:40:26 +00:00
smarcet
d6ca7262dd OpenstackId v3.0.12
Change-Id: I03ada0ba84005a88bcd9668f3c53c965ec9f5e24
Signed-off-by: smarcet <smarcet@gmail.com>
2020-06-11 11:54:42 -03:00
Clark Boylan
32ff621637 Cleanup old puppet management of release-volumes.py
This script has been moved into management done by ansible and is
executing on mirror-update not afsdb01. Cleanup the unused dead code.

Change-Id: Idc1c10cc968eef5ec1aeece70bad7606a7607269
2020-06-09 15:03:44 -07:00
Monty Taylor
8c9b4af143 Stop cloning more puppet modules
Previous review pointed out some additional modules we probably
aren't using any longer.

Remove the openafs::client section from openstack_project::server
because we're doing this with ansible now.

Depends-On: https://review.opendev.org/733890
Change-Id: Ib5104da9cf7d53b77191f48ec185f5d667d51944
2020-06-05 12:09:30 -05:00
Jeremy Stanley
f5f715008c Add missing HTTPS ports in ssldomains file
Some entries were added to the ssldomains list lacking a port
number. Add the HTTPS port to them.

Change-Id: I6bea5cbabb63ada9d817725e652157ccbdce7929
2020-05-19 20:32:46 +00:00
Ian Wienand
45201f3d66 Remove puppet mirror support
Remove the separate "mirror_opendev" group and rename it to just
"mirror".  Update various parts to reflect that change.

We no longer deploy any mirror hosts with puppet, remove the various
configuration files.

Depends-On: https://review.opendev.org/728345
Change-Id: Ia982fe9cb4357447989664f033df976b528aaf84
2020-05-16 10:14:25 +10:00
Ian Wienand
4233b79e31 Add limestone opendev.org server
This is to replace the puppet managed openstack.org server

Change-Id: I0e3586befd922cb56d1a0ec9c9cb650add9b225d
Depends-On: https://review.opendev.org/728314
2020-05-16 10:14:25 +10:00
Ian Wienand
a864212b1b Add vexxhost opendev.org mirrors
These are to replace the puppet-based openstack.org mirrors

Depends-On: https://review.opendev.org/728308
Change-Id: Ibdce99daa514fb445f1f8389e7c052ee151057ea
2020-05-16 10:14:25 +10:00
Clark Boylan
a83a763644 Add meetpad to cacti and ssl certcheck
People are starting to use this service so having performance metrics
over time is a good thing. We also want to avoid having our cert expire
unexpectedly.

Change-Id: I744b3e68f8f483b36c0d8ecb6f6f46a484a3577a
2020-05-15 13:51:33 -07:00