15161 Commits

Author SHA1 Message Date
Clark Boylan
f8bb46d810 Restore setup-org.yaml
This tasks file was removed when we rewrote the gitea management into
python. Unfortunately our gitea rename project tasks rely on this tasks
file to create new orgs if they are needed to rename a project.

Restore this file so that we can rename projects in this manner. Note we
move it into playbooks/ and rename it to gitea-rename-setup-org.yaml to
make it clear that the gitea management role doesn't use this set of
tasks, only the gitea project rename playbook does.

Change-Id: I782f6e56cad18bdcbf9504d51af15caa950a5752
2019-09-18 12:40:19 -07:00
James E. Blair
01bef804de Use the gerrit driver for the googlesource connection
This is the connection for Gerrit's Gerrit.  We can connect to it
directly with the Gerrit driver now that we support entirely
HTTP-based Gerrits and we have an account.  Use that so we can
set up some third-party CI testing.

Change-Id: I10e279218de985a0c17ea6b5bc46f3eebefcfb39
2019-09-17 14:21:16 -07:00
Zuul
64e5f9235d Merge "Mirror OBS ha-clustering repo for CentOS 7" 2019-09-17 11:58:03 +00:00
Mark Goddard
6b594d2e4d Remove #openstack-kayobe IRC channel
This will be merged with #openstack-kolla.

Change-Id: Ia66fd53ef70132cdcb67b881563b5b8402367efd
2019-09-17 10:53:05 +01:00
Radosław Piliszek
dc2510aac7 Mirror OBS ha-clustering repo for CentOS 7
Kolla uses this to build hacluster images.
Direct usage is causing timeouts recently.

I changed the formatting to make it more readable
and slapped a comment note for maintainers.

Change-Id: I68d7155718c0ae0744198ca96aca1a207bab7ed6
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-17 10:25:57 +02:00
Zuul
823298b365 Merge "Fix files matcher and bazel for gerrit base image" 2019-09-17 00:41:59 +00:00
Monty Taylor
072fcca06f Fix files matcher and bazel for gerrit base image
Use latest bazel

It seems 0.27 is now too old. This is what happens when I go on vacation
apparently.

Add in a hack to override the bazelversion. We'll remove this once
https://gerrit-review.googlesource.com/c/gerrit/+/237495 lands and
has been merged up.

Change-Id: Ib7a6d33ce8bf8498fd5cd09b25087dc09acb8df4
2019-09-16 21:20:18 +02:00
Zuul
65a16093c9 Merge "Add docs for recovering an OpenAFS fileserver" 2019-09-13 18:27:15 +00:00
James E. Blair
87fccc8e9b Add docs for recovering an OpenAFS fileserver
This should be a smooth recovery process.

Change-Id: I3c68b077e38a88160286d94e71676c0c4dbb6a51
2019-09-13 10:42:17 -07:00
Zuul
456a74134b Merge "fedora mirror update : add sleep" 2019-09-13 07:15:46 +00:00
Jeremy Stanley
72b3e36bf3 Add several missing ssldomains to certcheck config
There have been several Web sites added to files.o.o which missed
getting x509 SSL/TLS certificate checking added through our
certcheck cron job. Add those now so we know in advance whether
they're at risk of expiration.

Change-Id: I3eda77f165348e510d43344b172cf5b56ce2b003
2019-09-11 18:34:36 +00:00
Ian Wienand
3dcfe61d2f fedora mirror update : add sleep
As described inline, this should make our mirror pulses more robust
against timeouts.

This is probably ripe for turning into more of a library situation for
all the other "vos release" calls too.  But one thing at a time ... I
think we test with this for a while to see if stability returns.

Change-Id: I041a290053e4e8ceba80785598a5945e5adcf6f1
2019-09-11 13:15:29 +10:00
Zuul
2ffd934819 Merge "Set zuul_work_dir for tox testing" 2019-09-09 22:36:06 +00:00
Zuul
f2095676d5 Merge "Recognize DISK_FULL failure messages (review_dev)" 2019-09-09 11:37:04 +00:00
Ian Wienand
912dff49e7 Set zuul_work_dir for tox testing
Setting this to system-config allows us to run the base tests as 3rd
party ci for projects like testinfra.

Change-Id: I2d15df154dcdc7c5da6c3326fbecec2146201164
2019-09-09 09:44:43 +10:00
James E. Blair
d284333363 Remove newlines in afs server params variable
This causes newlines to appear in the config file which causes the
server to fail to start which is the opposite of what this is
supposed to do.

Change-Id: I2ff7e8835878652b3a7cdc2f633d263b37aaa7e9
2019-09-06 11:20:15 -07:00
Zuul
97e6d1d4af Merge "Don't run replication on gerrit startup" 2019-09-05 18:55:21 +00:00
Ian Wienand
ace1c39c61 gitea: use random time for git gc run
Randomising the time of this job should help avoid a thundering herd
of I/O intensive operations in the gitea environment.

Change-Id: I035f7781a397665357b6d039b989ab9fe6a46b8a
2019-09-04 05:15:21 +10:00
Ian Wienand
c9bea2cfc4 fedora-mirror: Use rsync itemize output
We are debugging why the fedora mirror has started to take 8+ hours to
release.  There is suspicion that rsync is updating the metadata of a
lot of files, causing the incremental update to be more of a full
release.

If you read the man page carefully:

 At the single -v level of verbosity, this does not mention when a
 file gets its attributes changed.  If you ask for an itemized list of
 changed attributes (either --itemize-changes or adding "%i" to the
 --out-format setting), the output (on the client) increases to
 mention all items that are changed in any way

So we need "-i" to see these updates

Change-Id: Ia9cff1e126404bc5a3346e758a15314834aef409
2019-08-30 13:29:54 +10:00
Zuul
74edfefda0 Merge "Split out bazel builder and gerrit base image" 2019-08-29 21:53:19 +00:00
Zuul
1b14855a45 Merge "AFS server restart and audit logging : helper script" 2019-08-29 21:03:09 +00:00
Zuul
d8448d1459 Merge "Build gerrit with bazel 0.27.1 instead" 2019-08-28 15:10:25 +00:00
Zuul
3eb182e551 Merge "Remove the extra bazel options" 2019-08-28 14:42:03 +00:00
Zuul
fa3004c245 Merge "Update bazel flags to be more inline with upstream" 2019-08-28 14:14:37 +00:00
Zuul
41340b4646 Merge "Zuul docs: Remove obsolete note" 2019-08-28 05:58:02 +00:00
Andreas Jaeger
13ce6ffd69 Zuul docs: Remove obsolete note
This note is not needed anymore, remove it.

Change-Id: I708fc3653833f794a448d40e3df72626f282b747
2019-08-28 06:51:58 +02:00
Zuul
cc506f5889 Merge "Docs: Zuul v2 is dead" 2019-08-28 01:55:53 +00:00
Zuul
8e0a03f7cf Merge "Filter docker directories out of cacti" 2019-08-28 00:11:17 +00:00
Andreas Jaeger
a6aaaed6a3 Docs: Zuul v2 is dead
Update the Zuul v3 page with correct links and description of promote
pipeline and remove the now obsolete note - and follow it with renaming
the document to zuul.

Change-Id: I9c89cb56d4a318f3a234e7f2f08dabb46d0dfab6
2019-08-27 08:39:57 +02:00
Ian Wienand
5686508716 Filter docker directories out of cacti
Docker hosts report back mounts in container directories via snmp
storage queries

 # php -q /usr/share/cacti/cli/add_graphs.php --host-id=585 --snmp-field=hrStorageDescr --list-snmp-values
 Known values for hrStorageDescr for host 585: (name)
 ...
  /var/lib/docker/containers/05ed2dc...
  /var/lib/docker/containers/7cebed4...
  /var/lib/docker/containers/f452861...

Because these can keep changing, hosts just end up getting more and
more invalid graphs in their results (see gitea0X hosts in cacti at
the moment).

Filter out docker directories from the query

Change-Id: Ia1db628975e7a67ad531438ef85735abae1ce652
2019-08-27 15:54:49 +10:00
Ian Wienand
b617a08989 fedora-mirror-update: update logging
We're having trouble with this volume; this makes the logging more
helpful for humans trying to establish the order of things happening.

It turns down the tracing, puts timestamps on the logs messages, makes
the messages more descriptive, and makes sure to timestamp the output
of the vos commands so we can see how long each step took.

Change-Id: Ia832b7ef86350efcbbe7d3b957e820a8a39df3e1
2019-08-27 14:09:49 +10:00
Monty Taylor
9bb1c73139 Split out bazel builder and gerrit base image
There is a bunch of duplication which needs to be redone almost never.
Split those into their own images so we can run them once and reuse them.

Change-Id: I923d4bff96dae75eb52a1c271fa52d5ae79933a0
2019-08-26 11:26:23 +02:00
Monty Taylor
3a26ed621a Build gerrit with bazel 0.27.1 instead
0.27 is the first semi-stable release.

Change-Id: I7314c6500841ed13fdc10dc951761f5fde4b28af
2019-08-26 11:26:23 +02:00
Monty Taylor
56ceaf1c40 Remove the extra bazel options
We had some extra bazel options that don't seem to be necessary
anymore now that we are using upstream bazel options appropriately.

Retry the build a couple of times if it goes south, inside of the
build image. This should allow re-use of the cache the second time,
and if there is a temporary error, it should pick up and move
forward.

Change-Id: I5f304acb21fd3a4d40701fc0414ae0c424c838e5
2019-08-26 11:26:19 +02:00
Monty Taylor
df8a7e37f0 Don't run replication on gerrit startup
Full replication is very costly and makes gerrit restarts expensive
these days. Turn off replicate_on_startup.

Depends-On: https://review.opendev.org/678486
Change-Id: I31d81821c645697e72a8702c60e2482156e01bb0
2019-08-26 10:16:08 +02:00
Monty Taylor
80981a52c8 Update bazel flags to be more inline with upstream
During the Gerrit Hackathon, we learned some things about setting
bazel options. Use the ones recommended in upstream docs rather than
these. The outcome should be largely the same.

Change-Id: I32b4c567488f0739fb80f69dc881b9837803575c
2019-08-25 11:34:58 +02:00
Ian Wienand
f3def9b84a Run ansible jobs on bridge.yaml changes
We almost merged I7ed75d253857f86b68f67023af6897af4e1b4f50 which would
have broken production Ansible runs due to a issue with the upgraded
Ansible and listener syntaxes.  CI was picking this up, but the jobs
weren't running on this change (in this case, it was noticed in a
follow-on job that triggered the letsencrypt jobs to run).

Add this file to all ansible tests so that if we bump versions of
ansible/openstacksdk/ara etc, we run all the tests in the gate.

Change-Id: I738c4e7721bd126e8e109c5ea1f38eba9e07b22b
2019-08-22 11:41:56 +10:00
Zuul
e03a97de8f Merge "files.o.o : publish .log as text/plain" 2019-08-22 00:34:44 +00:00
Zuul
a934f91f17 Merge "run_all.sh : add backup playbook" 2019-08-20 20:18:39 +00:00
Zuul
8303a60499 Merge "ircbot: add #openstack-ansible-sig to bots" 2019-08-20 13:56:13 +00:00
Clark Boylan
a248dbae63 Add more fedora-atomic mirror exclusions
We don't need things like efi configs or pxeboot configs or ISOs.
Exclude these items to further reduce the size of the fedora mirror.

Change-Id: I93003b2f48d79dae627026e2c7af748ea9a9e34d
2019-08-19 15:53:19 -07:00
Ian Wienand
0751b3d481 Convert nested bridge.o.o ARA report to static HTML
With the switch to swift logging, we need to convert the nested ARA
report to static HTML

Change-Id: I9e177915099598d5d48a31c15bd6db49e4d1c7e8
2019-08-19 10:28:57 +10:00
Mohammed Naser
66dc2a2449 ircbot: add #openstack-ansible-sig to bots
This change will add our IRC bots to the newly formed Ansible SIG,
in addition, it removes #dox in order to stay within the 120
channel limit.

The dox project has not seen a commit in 2 years and the channel
is pretty much empty/abanonded.

Change-Id: I3acd3cb77a9f71eb80921f4cbf2162113f40efea
2019-08-15 10:28:32 -04:00
Zuul
309daf9482 Merge "backup: minor fixes" 2019-08-12 23:43:29 +00:00
James E. Blair
859e386103 Add web_root option to zuul.conf
This is the base url that will be used for log reporting.

Depends-On: https://review.opendev.org/675655
Change-Id: Ia92a34e9ed506931e0d736ac034f60f4f7c381fc
2019-08-09 16:13:35 +00:00
Zuul
1c29fffa1a Merge "Replace wiki-dev02 with wiki-dev03" 2019-08-09 14:42:37 +00:00
Ian Wienand
376915e17a run_all.sh : add backup playbook
The backup roles have been debugged and are ready to run.

A note is added about having the backup server in a default disabled
state.  This was discussed at an infra meeting where consensus was to
keep it disabled [1].

[1] http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-06-11-19.01.log.html#l-184

Change-Id: I2a3d2d08a9d1514bf6bdcf15bc5bc95689f3020f
2019-08-09 16:43:55 +10:00
Ian Wienand
445eb7a7b2 backup: minor fixes
The ssh config file is /.ssh/config (not ssh_config)

We are accepting the ed25519 key, not the ecdsa key, so fix that in
the known_hosts stanza.

Change-Id: If3a42a7872f5d5e7a2bf9c3b5184fb14d43e6a1a
2019-08-09 14:11:41 +10:00
Zuul
a141ef8644 Merge "Add review-dev as a new backup client" 2019-08-09 00:04:42 +00:00
Zuul
a5378dfd1b Merge "Collect gitea sshd logs" 2019-08-08 18:12:49 +00:00