22 Commits (2c194e5cbf0b0bc0486bae5fcf3e294021369964)

Author SHA1 Message Date
Clark Boylan 5fa69476c4 Continue to add_host here even though we do it in base-jobs
We need to add host (and possibly the ssh host key so its here too) in
this playbook because the add_host from the base-jobs side is only
applicable to the playbook running in base-jobs. When we start our
playbook here that state is lost. Simple fix, just add_host it again.

Change-Id: Iee60d04f0232500be745a7a8ca0eac4a6202063d
3 years ago
James E. Blair b9f7f5506f Use infra-prod-base in infra-prod jobs
This uses a new base job which handles pushing the git repos on to
bridge since that must now happen in a trusted playbook.

Depends-On: https://review.opendev.org/742934
Change-Id: Ie6d0668f83af801c0c0e920b676f2f49e19c59f6
3 years ago
Monty Taylor 32978fe729 Update to tip of master in periodic jobs
The intent of the periodic jobs is to run with latest master. If
they get enqueued, then other patches land, they'll still run with
the value of the zuul ref from when they were enqueued. That's not
what we want for prod, as it can lead to running old versions of
config.

We don't usually like doing this, but in this case, rather than
making us remember to add a flag every time a prod job gets added
to a periodic pipeline, how's about we just calculate it.

Change-Id: Ib999731fe132b1e9f197e51d74066fa75cb6c69b
3 years ago
Monty Taylor 34c8e161b7 Set ansible_host explicitly
Pass the ansible_host variable explicitly to mirror-workspace-git-repos
because for some reason it's confused and getting localhost.

Change-Id: I8a30b98a6eef168d11d4d580de359546ee1da252
3 years ago
Monty Taylor 80222eabbd Remove ansible_user_dir
Put this in in the last patch without a specific need to. But
then we're getting an error. Because of course we are.

Change-Id: I5c982af2e1ba09a78162b2786e31f541247fce21
3 years ago
Monty Taylor 8fc86cbd7f Add port and user_dir to add_host in prod playbook
The mirror-workspace-git role expects things like ansible_port to
be set, but we're not producing them in our add_host command.

Change-Id: Ib80062736e91f8d1471a42edecdebb449f073927
3 years ago
Zuul 27f6013822 Merge "Switch to prepare-workspace-git" 3 years ago
Monty Taylor ebae022d07 Use project-config from zuul instead of direct clones
We use project-config for gerrit, gitea and nodepool config. That's
cool, because can clone that from zuul too and make sure that each
prod run we're doing runs with the contents of the patch in question.

Introduce a flag file that can be touched in /home/zuulcd that will
block zuul from running prod playbooks. By default, if the file is
there, zuul will wait for an hour before giving up.

Rename zuulcd to zuul

To better align prod and test, name the zuul user zuul.

Change-Id: I83c38c9c430218059579f3763e02d6b9f40c7b89
3 years ago
Monty Taylor 187177a9b2 Switch to prepare-workspace-git
This is what we actually want here and is more efficient as it
will use git push to push refs.

Change-Id: Ie0978bcd3c03c0a7905eb5c07246717806f5e3ae
3 years ago
Monty Taylor edd46d1acc Just use synchronize to sync the repos
We don't have cached repos, and our repos aren't so big
that we want to care about the git push difference.

Also - dont do delete: true like prepare-workspace does,
because deleting and then re-pushing project-config depending
on job would be costly.

Change-Id: I4c7bbc797f9f81878424b7bf2b7e83ec756de108
3 years ago
Monty Taylor cb58a10d5b Run playbooks out of zuul checkout
Instead of running from system-config, run from the zuul prepared
git repo state. We already have a mutex of one, so we'll never
be fighting. This lets us land stacks of changes and be sure they
will accurately always use the correct git state.

As a todo, we should update manage-projects to do the same with
project-config.

Change-Id: I358554e344e12561d1f3063e0724f6b61d1f15a7
3 years ago
Monty Taylor 1e79b848cd Rename ansible_forks to infra_prod_ansible_forks
ansible_forks is a magic var, so when we try to set it ansible
is overriding it.

Change-Id: I3ae45b28d545571f04b02927c3f6c97f145743bc
3 years ago
Clark Boylan dd2ba436f9 Become root when fixing bridge logging
We are writing to /var/log/ansible which needs root perms. This was not
done and the writes failed. Fix that.

Change-Id: Ibe93519f2f549e85f0e238a210999c6281f42ce6
3 years ago
Zuul 960b9d7bbd Merge "Improving logging of prod playbook jobs" 3 years ago
Clark Boylan c5263d76c9 Improving logging of prod playbook jobs
This updates prod playbook jobs to curate a set of logs on bridge if we
aren't publishing them to zuul. This way we have history on the bastion
server.

Change-Id: I73889754155298a8554ddc17bb413ae7764b9eae
3 years ago
Monty Taylor b0c0252f55 Remove leftover /var/run dir creation
This was added for a previous unlanded version of a change and
never removed. Oops.

Change-Id: I6006198dafac611c8ae1e2a6efcaddf5b9a60c8d
3 years ago
Monty Taylor b099b439b4 Grab logs in the case of errors
More importantly, put the log collection in an always
section of block, otherwise we won't get logs if a
playbook fails, which is pretty much exactly when we
want to get logs.

Change-Id: Ia8e581e522f75a5f5945bc2143eec63b93381a94
3 years ago
Monty Taylor 179d1a2516 Collect production playbook output
We need to log to a file and then collect it to the log output on
zuul. Default to true to that steady-state reads nicely. When we
add new jobs we should make sure to set to false first so that we
can vett the output before publishing it.

Change-Id: Ia4f759b82a5fff6e36e4284c11281254c0d5627d
3 years ago
Paul Belanger e2c4d9b3ad Set ansible_python_interpreter for bridge.o.o
We don't have python2 on bridge.o.o, force python3.

Change-Id: Ie8eb68007c0854329cf3757e577ebcbfd40ed8aa
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
4 years ago
Clark Boylan 9342c2aa6d Add zuul user to bridge.openstack.org
We want to trigger ansible runs on bridge.o.o from zuul jobs. First
iteration of this tried to login as root but this is not allowed by our
ssh config. That config seems reasonable so we add a zuul user instead
which we can ssh in as then run things as root from zuul jobs. This
makes use of our existing user management system.

Change-Id: I257ebb6ffbade4eb645a08d3602a7024069e60b3
4 years ago
James E. Blair 90e6088881 Configure adns1.opendev.org server via ansible
Change-Id: Ib4d3cd7501a276bff62e3bc0998d93c41f3ab185
5 years ago
Clark Boylan 0e9d830a64 Add job to run playbook on bridge.o.o
This new job is a parent job allowing us to CD from Zuul via
bridge.openstack.org. Using Zuul project ssh keys we add_host bridge.o.o
to our running inventory on the executor then run ansible on bridge.o.o
to run an ansible playbook in
bridge.openstack.org:/opt/system-config/playbooks.

Change-Id: I5cd2dcc53ac480459a22d9e19ef38af78a9e90f7
5 years ago