System configuration for the OpenDev Collaboratory
Go to file
Ian Wienand 4918594aa4 rsync-mirrors: drop rsync -t and make flags consistent
After investigating why every release seems to take hours, we noticed
that even in zero-delta updates where nothing has changed upstream,
we'd get a utimensat() call that updated the modified time nanoseconds
portion; e.g.

 lstat("Everything/x86_64/Packages/a/arch-install-scripts-23-1.fc31.noarch.rpm", {st_mode=S_IFREG|0644, st_size=28004, ...}) = 0
 utimensat(AT_FDCWD, "Everything/x86_64/Packages/a/arch-install-scripts-23-1.fc31.noarch.rpm",
  [UTIME_NOW, {tv_sec=1585676005, tv_nsec=6041000} /* 2020-03-31T17:33:25.006041000+0000 */], AT_SYMLINK_NOFOLLOW) = 0

This does not apply; in fact openafs uses the ns field as some sort of
generation counter [1].  This update is enough to convince openafs the
file has changed and it needs to be resynced, meaning that basically
every rsync run results in a full release.

This unnecessary update been fixed with [2] but is only in rsync
3.1.3+; our bionic host is currently 3.1.2.  Dropping "-t" from the
rsync commands avoids transferring modification times and should avoid
this problem.

While looking, "-D" turns on "--devices" and "--specials" to transfer
block devices and named sockets/fifos.  Turn this off.

Also remove "-p" if it was present.  We already did this for centos
with Ib5db052cdd23e39aecbeead15cf08d4bd7fcab38 and Fedora with
Id24196791f80cd99fe8a330fb2c7c6d893fc9995, where odd upstream
permissions such as setgid on directories can't be synced to afs.
Consistently remove it.

Also switch back the fedora updates to just "-v"; we had it at "-i"
for debugging.

[1] http://eavesdrop.openstack.org/irclogs/%23opendev/%23opendev.2020-06-15.log.html#t2020-06-15T02:58:08-2
[2] https://git.samba.org/?p=rsync.git;a=patch;h=0f8e9e2d8638e47d646a6baba694b303ac84e695

Change-Id: I78f3d4990b168c71185eb1c4900ceeaca4b6a16f
2020-06-16 11:50:48 +10:00
doc Add bit more info on disabling ansible runs 2020-06-15 14:41:51 -05:00
docker Merge "Be explicit about using python3 in docker images" 2020-06-15 23:37:17 +00:00
hiera Revert "Add three more JVB servers" 2020-06-05 20:47:17 +00:00
inventory Merge "Add Zookeeper TLS support" 2020-06-15 21:48:14 +00:00
kubernetes Update opendev git references in puppet modules 2019-04-20 18:26:07 +00:00
launch Merge "Change launch scripts to python3 shebangs" 2020-06-15 21:14:26 +00:00
manifests Cleanup old puppet management of release-volumes.py 2020-06-09 15:03:44 -07:00
modules/openstack_project Merge "Cleanup old puppet management of release-volumes.py" 2020-06-15 21:40:26 +00:00
playbooks rsync-mirrors: drop rsync -t and make flags consistent 2020-06-16 11:50:48 +10:00
roles Merge "ARM64 openafs role tests" 2020-06-09 23:53:44 +00:00
roles-test Remove Puppet 5 testing 2020-06-09 10:15:05 +10:00
testinfra Merge "Add Zookeeper TLS support" 2020-06-15 21:48:14 +00:00
tools Switch prep-apply.sh to use python3 2020-06-15 14:43:25 -05:00
zuul.d Merge "Fake zuul_connections for gate" 2020-06-15 21:47:49 +00:00
.ansible-lint Go back to using find with ansible-lint 2020-03-11 16:30:33 -05:00
.gitignore Ignore ansible .retry files 2016-07-15 12:04:48 -07:00
.gitreview OpenDev Migration Patch 2019-04-19 19:26:05 +00:00
COPYING.GPL Add yamlgroup inventory plugin 2018-11-02 08:19:53 +11:00
Gemfile Update some paths for opendev 2019-04-20 09:31:14 -07:00
README.rst Update some paths for opendev 2019-04-20 09:31:14 -07:00
Rakefile Further changes to bring puppetboard online 2014-03-22 12:54:38 -07:00
bindep.txt Add libffi dev packages needed for ansible install 2016-10-04 15:20:00 -07:00
install_modules.sh Merge "Better checking for tags when cloning puppet modules" 2020-01-16 23:01:33 +00:00
install_puppet.sh Install the puppetlabs puppet package 2018-08-23 14:55:08 +10:00
make_swap.sh Use swapfile if no extra device is present 2019-07-17 10:37:30 -07:00
modules.env Stop cloning more puppet modules 2020-06-05 12:09:30 -05:00
mount_volume.sh Add support to launch-node for cinder attach 2016-04-19 11:07:23 -07:00
run_k8s_ansible.sh Invoke run_k8s_ansible from its directory 2019-05-07 16:03:59 -07:00
run_puppet.sh Clean up bashate failures 2014-09-30 12:40:59 -07:00
setup.cfg Mention new mailing lists 2020-04-06 18:19:28 +00:00
setup.py Update to openstackdocstheme 2018-06-25 11:19:43 +10:00
test-requirements.txt Update to ansible-lint 4.1.0 2019-06-06 22:13:12 +00:00
tox.ini Split inventory into multiple dirs and move hostvars 2020-06-04 07:44:36 -05:00

README.rst

Puppet Modules

These are a set of puppet manifests and modules that are currently being used to manage the OpenStack Project infrastructure.

The main entry point is in manifests/site.pp.

In general, most of the modules here are designed to be able to be run either in agent or apply mode.

These puppet modules require puppet 2.7 or greater. Additionally, the site.pp manifest assumes the existence of hiera.

See http://docs.openstack.org/infra/system-config for more information.

Documentation

The documentation presented at http://docs.openstack.org/infra/system-config comes from https://opendev.org/opendev/system-config repo's docs/source. To build the documentation use

tox -evenv python setup.py build_sphinx