Commit Graph

10459 Commits

Author SHA1 Message Date
David Shrewsbury
b0e2df07b4 Remove shrews from infra-root
Change-Id: I55d9670f216fb6d36be8ec080fcc02e40bf83a68
2020-03-26 13:38:02 -04:00
Ian Wienand
dcc23be55b Remove static site puppet
Remove the puppet related to the static server

Story: #2006598
Task: #36758

Change-Id: I744da83d811e715ff8049eacf3013cd4831e1c49
2020-03-26 10:36:16 +11:00
Ian Wienand
f55580fbf0 Remove files02.openstack.org and related puppet
All this has moved to static.opendev.org; the server can now be
removed.

Change-Id: I8ca5d7a206e950c28bb8372a85b6a62d6b9ba00c
2020-03-26 10:36:13 +11:00
smarcet
de04c6da67 Added new param to IDP config
Added user_spam_processor_to to dev/production

Change-Id: I36a41bdcd23df9540183da5a087d364778d60b48
Signed-off-by: smarcet <smarcet@gmail.com>
2020-03-25 11:32:17 -03:00
Zuul
5b6d2fb4e9 Merge "OpenStackID 3.0.6 production deployment" 2020-03-24 20:50:37 +00:00
Zuul
3af9548401 Merge "Update CLA link" 2020-03-21 18:24:50 +00:00
Zuul
06e0155cb8 Merge "Update infra-manual location" 2020-03-21 18:02:17 +00:00
Andreas Jaeger
4293e228b7 Update CLA link
link to contributor guide, the OpenDev manual does not talk about CLA
anymore.

Change-Id: Ia414513fc1645e0eec462f95af0d9247eb4c0a99
2020-03-21 18:17:57 +01:00
smarcet
055796bb4e OpenStackID 3.0.6 production deployment
Change-Id: Ib4621b38d78fb61311a65dd3878a0a61e799f367
Signed-off-by: smarcet <smarcet@gmail.com>
2020-03-21 12:47:59 -03:00
Andreas Jaeger
2c0b82e5e8 Update infra-manual location
The infra-manual now lives on docs.opendev.org, update links.

New location is: https://docs.opendev.org/opendev/infra-manual/latest

Change-Id: I7716c68cbff4f3a640d7161f59cfc034a7ccca52
2020-03-20 22:03:09 +01:00
smarcet
d138243bd5 OpenStackId production release 3.0.5
Change-Id: I8d2e2d96ba3543de6b47c5001f9bc2ebab115286
Signed-off-by: smarcet <smarcet@gmail.com>
2020-03-20 17:28:04 -03:00
Zuul
b0f81dc7b9 Merge "Update git.starlingx/git.airship redirects" 2020-03-19 01:21:21 +00:00
Zuul
51a5f5488f Merge "Update git.zuul-ci.org redirects" 2020-03-19 01:21:20 +00:00
Andreas Jaeger
eecf3e71fc Update git.starlingx/git.airship redirects
After the big OpenDev rename, these repos got renamed again. Update the
redirects for git.airshipit.org and git.starlingx.io to point to the
current location.

Update test_static.py for this, change the test repo since
airship-in-a-bottle was first renamed to in-a-bottle and later to
airship-in-a-bottle.

Change-Id: I71b786cd528aac9ae68464618db02e22cd4c0b5b
2020-03-18 18:39:48 +01:00
Andreas Jaeger
a6480bcefb Update git.zuul-ci.org redirects
zuul and nodepool now life in opendev, avoid double redirects and
redirect directly to final location.

Change-Id: Ia55d76b24f07ec64cb55055955c4549f3706a95b
2020-03-18 18:28:42 +01:00
Jeremy Stanley
4cbdc2fc4d Set up inventory and cert for Open Edge mirror
This adds the Open Edge (formerly Fortnebula) CI mirror.

Change-Id: I1ccf2a602f8a41e00bc64a9516a326cc07d9b254
Depends-On: https://review.opendev.org/711787
2020-03-07 00:24:20 +00:00
Jeremy Stanley
43ed9fc297 Moving FortNebula to OpenEdge
Sister change for Ia5caff34d3fafaffc459e7572a4eef6bd94422ea and
removing earlier references to the mirror server in preparation for
building and adding the new one.

Change-Id: I7d506be85326835d5e77a0c9c461f2d457b1dfd3
2020-03-06 20:43:56 +00:00
Zuul
a70c4a1cce Merge "Pin lodgeit to the current version" 2020-03-05 17:50:27 +00:00
Monty Taylor
b8f9cc40aa Pin lodgeit to the current version
So we can be careful about landing lodgeit changes.

Depends-On: https://review.opendev.org/711344
Change-Id: If1ae66fc94d5ceed458b93cb10f0bb061df85021
2020-03-04 23:54:41 +00:00
Zuul
1cae4d56a0 Merge "ssl cert check: add recent domains" 2020-03-04 00:57:19 +00:00
Ian Wienand
bd3b24fcc4 ssl cert check: add recent domains
Add some domains added recently
(https://review.opendev.org/#/c/710160/)

Change-Id: I8d2671494ddf9097449464df1b538e699fa5c1be
2020-03-04 00:09:46 +00:00
Clark Boylan
61caec5b77 Use LE cert on review.open*.org
We previously had two manually issued certs (one each for opendev.org
and openstack.org) but now have a single cert with all the appropriate
names in it automatically issued by LE. Use this new cert before the old
one expires.

Change-Id: I635d2bfd820fe138ee951833dd66f157b2b7c097
2020-02-28 08:10:24 -08:00
smarcet
eed9116625 OpenStackId v3.0.4 Deployment
Fixed inconsistent ORM mappings

Change-Id: I1806f0c22a21ebf495aa725998ba0e0e57bf3b4a
Signed-off-by: smarcet <smarcet@gmail.com>
2020-02-26 16:58:58 -03:00
smarcet
19662d7d3f OpenStackId v3.0.3 Deployment
Change-Id: I239f8311139d59260430baee0c6cb910c31caf3f
Signed-off-by: smarcet <smarcet@gmail.com>
2020-02-25 22:21:20 -03:00
Zuul
9566bd0e2a Merge "Revert "Generate list of 404s for docs.o.o"" 2020-02-25 23:39:02 +00:00
smarcet
88b26edf24 OpenStackId v3.0.1 deployment
Change-Id: I8a49dbde1bbcd935b380f003160040d634b2062c
Signed-off-by: smarcet <smarcet@gmail.com>
2020-02-25 15:35:17 -03:00
smarcet
09f35f14bf OpenStackID v3.0.0
* migrated user storage to idp
* created users crud
* created groups crud
* migrated from eloquent to doctrine orm
* reafactoring

Change-Id: I766bbb75c0e65f504880e8c59951f63494a1e13f
Signed-off-by: smarcet <smarcet@gmail.com>
2020-02-25 09:45:06 -03:00
smarcet
ce6de87ea7 Removed OpenStackID stale config variables
Removed all variables related to Silverstripe
Dependency

Change-Id: Ib5e6834686c4952dd8e7220a31abe71a9278e397
Signed-off-by: smarcet <smarcet@gmail.com>
2020-02-24 14:26:30 -03:00
Ian Wienand
55da1e3d06 Revert "Generate list of 404s for docs.o.o"
This reverts commit c25e91f496.

This script parses the Apache logs and writes out a local count of the
404 data to files.openstack.org, and then exports it via
files.openstack.org.

As part of the spec [1] we're trying to remove publishing from local
volumes, in general.

Since this is not widely used, there is only one link to it, it's not
discoverable from the landing page of files.openstack.org (which just
shows the afs directory listing), it has a very long latency making it
not that useful for debugging and grepping the logs there have been no
accesses in the past 2 weeks (as far back as logs go) I propose we
remove it.

If we want to retain this, we should publish the output alongside the
docs AFS volume.  That could certainly be done by distributing the
docs keytab to the host and having it write out in a similar cron job.
Another option could be to setup a keypair for remote login and keep
that as a secret in Zuul, and do the same from a periodic job
(complicated by apache logs being root only, so needs some sudo magic
or similar).  Or, we could figure out an altogether better, privacy
respecting client analytics solution.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/retire-static.html

Depends-On: https://review.opendev.org/709036
Change-Id: Iccf24a72cf82592bae8c699f9f857aa54fc74f10
2020-02-24 14:43:11 +11:00
Zuul
30297fb10d Merge "Migrate AFS publishing to mirror-update.opendev.org" 2020-02-12 22:40:43 +00:00
Ian Wienand
97c4735129 Move afsmon to mirror-update.opendev.org
This migrates the afsmon script from puppet deploying on
mirror-update.openstack.org to ansible deploying on
mirror-update.opendev.org.

There is nothing particularly special and this just a straight install
with some minor dependencies.  Since we have log publishing running on
the opendev.org server, we publish the update logs alongside the
others.

Change-Id: Ifa3b4d59f8d0fc23a4492e50348bab30766d5779
2020-02-12 14:38:48 +11:00
Ian Wienand
6991905e52 Migrate AFS publishing to mirror-update.opendev.org
This follows-on from I62ae941e70c7d58e00bc663a50d52e79dfa5a684 to
remove the old publishing job from the afs server and enable live
publishing in the cron job on mirror-update.opendev.org.

Change-Id: Ib6cc094e6f02b513c5218264657cec9064fe867c
2020-02-11 16:14:43 +11:00
Jeremy Stanley
bc0480756c Add missing Lets Encrypt certs to ssldomains list
A bunch of the sites we've started managing LE certs for are not
getting their expirations checked, so fix that. In particular, sites
recently moved off the multi-domain SAN cert for the old
static.openstack.org server (omitted the logs site as it's
deprecated), and many of the rebuilt CI mirrors (with the exception
of mirror01.gra1.ovh.opendev.org which is presently in a SHUTOFF
state for unknown reasons). Also add graphite which was previously
missed, and review-dev because we can now that it's no longer
sporting snakeoil.

When this merges, we're also going to start getting alerts for an
expired cert on mirror.gra1.ovh.opendev.org, unless someone gets a
chance to look into it first.

Change-Id: I98a98e0d2ff081c51c33d980274f3ee8c0266802
2020-02-07 15:29:50 +00:00
Clark Boylan
bd752a0bfe Keep only 7 days of records in ElasticSearch
We have been running out of disk recently with some indexes requiring
more than 400GB of space per index replica. Actual disk space
requirements are double that as we run with a replica. On top of that
the idea is that 5 of 6 elasticsearch nodes have enough space for all
our data so that we are resilient to losing a node.

Napkin math:

  400 * 10 * 2 = ~8TB of disk
  400 * 7 * 2 = ~5.6TB of disk

Each of the six ES nodes has 1TB of disk allocated to ES so 5.6TB should
get us just under the limit. Then for handling a node outage weekends
tend to not have as many records so our actual usage should be a little
lower.

Change-Id: Ie677bd47a9886870bc83876d2407742133299861
2020-02-06 13:50:56 -08:00
Clark Boylan
8b51cc616a Copy unminimized flot files
yui-compressor is not happy with flot's jquery.flot.js file. These files
are actually pretty small especially when compared to our input json
data. Lets just serve them as is.

Depends-On: https://review.opendev.org/704716
Change-Id: Ibfd081bb73a6c352798a7822ab781c972ace4bc3
2020-01-28 16:41:07 -08:00
Clark Boylan
89b15fec21 Flot sources moved, update our minimization to accomodate
Flot sources moved from the repo top level dir into the source dir.
Accomodate this when we minimize and copy those js files.

Change-Id: I3522271361fc43550ac1c6dc2a690c5cc5ce9c64
2020-01-27 15:38:39 -08:00
Zuul
e00dd724c6 Merge "Add mailing list for OpenInfra Labs" 2020-01-21 17:20:41 +00:00
Mohammed Naser
4b90ba9b4f Add mailing list for OpenInfra Labs
This creates a new mailing list under OpenDev for discussion
around OpenInfra Labs.

Change-Id: I6754df15b7cd205fc9a9d6bc1ace5cbcd65faa84
2020-01-17 13:03:35 -05:00
Zuul
06096940e2 Merge "Revert "Enable gerrit replication.autoReload for review"" 2020-01-15 22:56:22 +00:00
Zuul
6489d40f7e Merge "Periodically release additional AFS volumes" 2020-01-15 17:05:15 +00:00
Monty Taylor
6f3a2792cc Switch to ansible on review-dev
The review-dev service playbook should do everything now that
the puppet did. Update how we're running things.

Change-Id: I70303c48328ea6713c24bf9c6f63d4808d30b95c
2020-01-14 12:04:15 -06:00
Jeremy Stanley
eec87262b2 Add arm64 to reprepro updates file for UCA
This was missed in the earlier change to add arm64 to the template
for individual distributions.

Change-Id: I0fd24fd6c3b48c2874c7f5690709fed20d1547cf
2020-01-09 21:35:05 +00:00
Zuul
abf455e22b Merge "Use LE cert for git.zuul-ci.org" 2020-01-09 18:19:25 +00:00
Zuul
fe487524a9 Merge "Add insecure-ci-registry to ssldomains" 2020-01-09 01:01:58 +00:00
Zuul
85a4561536 Merge "Alphabetize ssldomains file" 2020-01-09 01:01:56 +00:00
James E. Blair
447b8513ea Use LE cert for git.zuul-ci.org
The cert has been issued.

This partially reverts commit 42c0d0696c.

Change-Id: I5f6c9d648a6fdfc3f9464c01f51d95c5cd72ed49
2020-01-08 16:14:26 -08:00
Clark Boylan
42c0d0696c Fix zuul-ci.org vhost cert paths
We were setting the cert file contents to the paths rather than updating
the paths to point at the new LE certs. Fix this by setting the _file
vars which update the path.

This includes a partial revert of the previous change to not switch
git.zuul-ci.org over to LE as we haven't provisioned an LE cert for it
yet.

Change-Id: I41c2aa1d03afba4ebf6378e9abf8276154666df7
2020-01-08 10:03:05 -08:00
Zuul
44ca7193ea Merge "UCA: mirror Ussuri packages" 2020-01-08 16:26:29 +00:00
Marcin Juszkiewicz
dbdb8801f3 UCA: mirror Ussuri packages
Change-Id: Ia00b269deee91dbf681a2f6c025ff2691ff6ce3d
2020-01-08 15:58:33 +01:00
Jeremy Stanley
74dc37afaf Add insecure-ci-registry to ssldomains
The insecure-ci-registry.opendev.org service uses an X.509 cert on
5000/tcp, so we should track this to catch when it's going to
expire.

Change-Id: I5d18599e5b5b258ce158f964cb1ff95df6dc6d92
2020-01-08 00:17:39 +00:00