55 Commits

Author SHA1 Message Date
ramishra
c9991c2e31 Use 'wallaby' heat_template_version
With I57047682cfa82ba6ca4affff54fab5216e9ba51c Heat has added
a new template version for wallaby. This would allow us to use
2-argument variant of the ``if`` function that would allow for
 e.g. conditional definition of resource properties and help
cleanup templates. If only two arguments are passed to ``if``
function, the entire enclosing item is removed when the condition
is false.

Change-Id: I25f981b60c6a66b39919adc38c02a051b6c51269
2021-03-31 17:35:12 +05:30
Alexey Stupnikov
e7077e984b Remove or fix outdated/incorrect swift hieradata definitions
- swift::proxy::s3token::www_authenticate_uri doesn't exist,
  swift::proxy::s3token::auth_uri should likely be used
  instead.

Partial-Bug: #1916386
Change-Id: I48ada3704f96d858d79e31d4b1600c7905d1f62d
2021-03-21 16:53:12 +01:00
Zuul
b2d49f6e39 Merge "Add TLS support to services using memcached" 2021-03-18 05:15:23 +00:00
Grzegorz Grasza
1ceb521805 Add TLS support to services using memcached
This patch enables TLS connections to memcached in services which
support it. Specifically the settings are consumed by swift's internal
memcached client through puppet-swift; or oslo.cache, through
puppet-ceilometer, puppet-keystone, puppet-nova, puppet-heat
and puppet-oslo.

Depends-on: https://review.opendev.org/772685
Depends-on: https://review.opendev.org/761605
Depends-on: https://review.opendev.org/764764
Depends-on: https://review.opendev.org/764763
Depends-on: https://review.opendev.org/765100
Change-Id: Ic77ed56c32c7071ce126a1528030094b97894653
2021-03-16 15:28:49 +01:00
Giulio Fidente
c8a828aab2 Fix start order for {swift_proxy,glance_api}_tls_proxy
In stack updates adding TLS to Swift would not work because
swift_proxy_tls_proxy is started when the pre-existing
swift_proxy container is still up and listening on the TCP port
which should instead be used by swift_proxy_tls_proxy

This change ensures swift_proxy_tls_proxy and glance_api_tls_proxy
containers are started after the actual swift_proxy and glance_api
containers are started.

Co-Authored-By: Michele Baldessari <michele@acksyn.org>
Change-Id: I980dfc54b799c6b1d648489a727a590dd26fa502
Closes-Bug: 1918642
2021-03-11 12:36:10 +01:00
ramishra
7f195ff9a8 Remove DefaultPasswords interface
This was mainly there as an legacy interface which was
for internal use. Now that we pull the passwords from
the existing environment and don't use it, we can drop
this.

Reduces a number of heat resources.

Change-Id: If83d0f3d72a229d737a45b2fd37507dc11a04649
2021-02-12 11:38:44 +05:30
Takashi Kajinami
971438dd8d Simplify composition of swift-proxy-server pipelines
This change refactors the logic to genrate swift-proxy-server pipelines
by replacing current usage of yaql by native list_concat function.

Change-Id: Ia407b54ba11acdc788c349e2ff0e973f94d1d450
2020-12-24 09:02:45 +09:00
Zuul
d659e6fce2 Merge "Refresh Swift ring files without restarting containers" 2020-11-02 19:08:56 +00:00
Christian Schwede
cb982440d7 Refresh Swift ring files without restarting containers
Updating rings consists of multiple steps today:
1. Apply puppet in the swift_ringbuilder container
2. Copying files in step 3 using swift_copy_rings
3. Run kolla_set_config to copy files to /etc/swift before starting
   Swift service processes

This needs a container restart today, because kolla_set_config will be
executed only on container (re-)starts.

This patch will execute kolla_set_configs at the step 5 of deployment,
and apply all ring changes, if any, for the Swift processes without
a container restart. Swift processes will notice the changed ring files
within 15 seconds and will use the updated rings.

Co-authored-by: Bogdan Dobrelya <bdobreli@redhat.com>
Change-Id: Ibdd783b484a84c0fdfaac84d892a8ea46be85fde
2020-10-29 14:35:17 +01:00
Takashi Kajinami
37548ddb40 Enforce internal api for token verification
This change enforces the usage of internal api for token verification,
so that internal requests to keystone uses internal endpoint instead
of admin endpoint which is deployed on provisioning network by default.

Change-Id: I8b5ac36ff1da46844d18fa73f835175e52719a63
Closes-Bug: #1899266
2020-10-11 15:46:08 +09:00
Alex Schultz
cef9c0e88e Update swift rsyslog configuration
Rather than use the shell tasks to determine if we have rsyslog, we can
use ansible's systemd module. Additionally, we should only run this bit
of code once if proxy and storage are being installed on the same node.
Currently it'll run the same bit of code twice.

Change-Id: I6cac4725ad230869c1bbe655d5b9020fa4318a39
2020-09-08 14:29:47 -06:00
Jose Luis Franco Arza
8783ec9c45 Remove ffwd-upgrade leftovers from THT.
Now that the FFU process relies on the upgrade_tasks and deployment
tasts there is no need to keep the old fast_forward_upgrade_tasks.

This patch removes all the fast_forward_upgrade_tasks section from
the services, as well as from the common structures.

Change-Id: I39b8a846145fdc2fb3d0f6853df541c773ee455e
2020-07-23 15:33:25 +00:00
Michele Baldessari
bd4b57c269 Remove /run from some services
redis(non-pcmk), nova-scheduler and swift-proxy do not need /run bind
mounted from the host.  As a matter of fact bind-mounting /run is
problematic due to a number of reasons (see LP#1883849 for more
background). In particular swift-proxy is the only swift container
(out of 9) that has /run bind-mounted.

These three services always had /run from the very beginning:
- redis -> Ie750caa34c6fa22ca6eae6834b9ca20e15d97f7f
- nova-scheduler -> I39436783409ed752b08619b07b0a0c592bce0456
- swift-proxy -> I2d96514fb7aa51dffe8fe293bc950e0e99df5e94

Tested this by applying this patch on a train deployment and
deployed an undercloud and an overcloud with it.
Verified that:
A) /run:/run is not present in the three containers
B) Deploy of UC and OC worked correctly
C) Tempest still works
D) Restarting the swift_proxy and nova_scheduler works correctly
E) Reboot the overcloud worked okay and tempest still works after the
   full overcloud reboot
F) Ran a minor UC update
G) Ran a minor update on all nodes and tempest still worked
H) Ran a redeploy on all nodes and tempest still worked

NB: I did not investigate other containers that bind mount /run
because they 1) seem to need it and 2) had no means to do proper
testing.

NB2: Note that while once we rebuild containers with
I81e5b7abf4571fece13a029e25911e9e4dece673 this change here is not
strictly needed for the LP bug, but is a nice cleanup nonetheless.
So this is to be backported only if rebuilding containers is
a problematic/costly move

Change-Id: Ic1a892a7f78a54b5e149f5ce52cb9db68ebc9529
Related-Bug: #1883849
2020-07-11 07:03:24 +00:00
Emilien Macchi
1a48fa61f4 Sync httpd conf.modules.d configs
For containers which run httpd, make sure conf.modules.d is also synced
into the container; so apache doesn't fail with:
AH00534: httpd: Configuration error: More than one MPM loaded.

This is now required since:
6425cc46a8

Change-Id: Ib315d10dbdbbad1628f536a74cd1fca371f018f5
Closes-Bug: #1884115
2020-06-24 03:32:02 +00:00
Jose Luis Franco Arza
94bc023390 Add mode option when creating persistent directories.
Almost every single tripleo service creates a persistent directory. To
simplify the creation, a with_items structure was being used. In which
many times, the mode option was being set. However, that mode option
was not taken into account at the time of creating the file. As a
consequence, the directory was being created with its father directory
rights, instead of the ones being passed in the template.

Change-Id: I215db2bb79029c19ab8c62a7ae8d93cec50fb8dc
Closes-Bug: #1871231
2020-04-20 15:37:08 +02:00
Takashi Kajinami
fffdcf0f30 Use absolute name to include puppet classes
Current puppet modules uses only absolute name to include classes,
so replace relative name by absolute name in template files so that
template description can be consistent with puppet implementation.

Change-Id: I7a704d113289d61ed05f7a31d65caf2908a7994a
2020-04-11 08:13:23 +09:00
Zuul
3910afffb9 Merge "Remove useless auth_uri from ceilometer middleware" 2020-03-10 17:05:04 +00:00
Zuul
eaac8222e8 Merge "Add missing keystone region for swift user auth" 2020-03-10 17:05:02 +00:00
Emilien Macchi
38bad5283f Remove all ignore_errors to avoid confusion when debugging
- deploy-steps-tasks-step-1.yaml: Do not ignore errors when dealing
  with check-mode directories. The file module is resilient enough to
  not fail if the path is already absent.

- deploy-steps-tasks.yaml: Replace ignore_errors by another condition,
  "not ansible_check_mode"; this task is not needed in check mode.

- generate-config-tasks.yaml: Replace ignore_errors by another
  condition, "not ansible_check_mode"; this task is not needed in check mode.

- Neutron wrappers: use fail_key: False instead of ignore_errors: True
  if a key can't be found in /etc/passwd.

- All services with service checks: Replace "ignore_errors: true" by
  "failed_when: false". Since we don't care about whether or not the
  task returns 0, let's just make the task never fail. It will only
  improve UX when scrawling logs; no more failure will be shown for
  these tasks.

- Same as above for cibadmin commands, cluster resources show
  commands and keepalived container restart command; and all other shell
  or command or yum modules uses where we just don't care about their potential
  failures.

- Aodh/Gnocchi: Add pipefail so the task isn't support to fail

- tripleo-packages-baremetal-puppet and undercloud-upgrade: check shell
  rc instead of "succeeded", since the task will always succeed.

Change-Id: I0c44db40e1b9a935e7dde115bb0c9affa15c42bf
2020-03-05 09:22:04 -05:00
Takashi Kajinami
25c21dddf2 Remove useless auth_uri from ceilometer middleware
... because we don't have auth_uri parameter in ceilometer middleware,
and the hieradata for it was deprecated in puppet-swift.

Depends-on: https://review.opendev.org/#/c/707748/
Change-Id: I4e2430ca479cd2a3ef3bfa8bc32ff6cc153500c5
2020-02-14 11:37:38 +09:00
Takashi Kajinami
d334e58539 Add missing keystone region for swift user auth
Depends-on: https://review.opendev.org/#/c/707168/
Change-Id: Ia24b8de1421d26d411d0abe7f4bd7aa293611253
2020-02-13 11:18:27 +09:00
Zuul
b130f78076 Merge "Replace svirt_sandbox_file_t by container_file_t" 2020-02-10 13:58:31 +00:00
Zuul
c48ccacf74 Merge "Remove deprecated authtoken::auth_uri" 2020-02-07 17:43:51 +00:00
Cédric Jeanneret
0875895553 Replace svirt_sandbox_file_t by container_file_t
While they are, at SELinux level, exactly the same (one is an alias to
the other), the "container_file_t" name is easier to understand (and
shorter to write).

A second pass in a couple of days or weeks will be needed in order to
change files that were merged after this first pass.

Change-Id: Ib4b3e65dbaeb5894403301251866b9817240a9d5
2020-02-07 13:33:20 +01:00
Jesse Pretorius (odyssey4me)
2092b1303f Update ffwd-upgrade branch names
The next iteration of fast-forward-upgrade will be
from queens through to train, so we update the names
accordingly.

Change-Id: Ia6d73c33774218b70c1ed7fa9eaad882fde2eefe
2020-01-27 19:42:40 +00:00
Takashi Kajinami
8cc62c5f14 Remove deprecated authtoken::auth_uri
auth_uri parameter in authtoken was already removed from puppet modules[1],
so remove it from hieradata.

Also, some service templates missed www_authenticate_uri, which was
introduced as a replacement of auth_uri, so add it to make sure that
we have a correct parameter confugured.

[1] I12b4049e4942911c8d1d8027c579eb4c0d1a53eb

Change-Id: I1e8378f58662377344194916e8bc336df02d0591
2020-01-26 09:26:50 +09:00
Zuul
a3a7a25db3 Merge "Fix permission error if Barbican is enabled for Swift" 2020-01-09 21:28:03 +00:00
Christian Schwede
77b48f3988 Fix permission error if Barbican is enabled for Swift
The config files need to be relabled too avoid permission errors, otherwise
the Barbican key ID is not set and Swift fails to start.

Closes-Bug: 1858845
Change-Id: Idb94fee8aee1de3b1c7338108ccd8b45b6f169bf
2020-01-08 20:37:24 +01:00
Emilien Macchi
7f40baabcd Manage all Keystone resources with Ansible
Depends-On: I557d8f33c9c699aed14b3b6fc1d1c0407365cd08
Depends-On: Ia68f8852662fb4abbd194954a246afb740bf3f71

Change-Id: I96a3351fca26cd8bb122a86cb4c3a58d5f88573e
2020-01-06 22:33:05 +00:00
Sagi Shnaidman
016f7c6002 Remove unnecessary slash volume maps
When podman parses such volume map it removes the slash
automatically and shows in inspection volumes w/o slash.
When comparing configurations it turns to be a difference and
it breaks idempotency of containers, causing them to be recreated.

Change-Id: Ifdebecc8c7975b6f5cfefb14b0133be247b7abf0
2019-12-04 20:32:14 +02:00
Kevin Carter
50367fbe35 Convert firewall rules to use TripleO-Ansible
This change converts our filewall deployment practice to use
the tripleo-ansible firewall role. This change creates a new
"firewall_rules" object which is queried using YAQL from the
"FirewallRules" resource.

A new parameter has been added allowing users to input
additional firewall rules as needed. The new parameter is
`ExtraFirewallRules` and will be merged on top of the YAQL
interface.

Depends-On: Ie5d0f51d7efccd112847d3f1edf5fd9cdb1edeed
Change-Id: I1be209a04f599d1d018e730c92f1fc8dd9bf884b
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2019-11-18 15:40:22 -06:00
Jose Luis Franco Arza
4cbae84c75 Get rid of docker removing in post_upgrade tasks.
When upgrading from Rocky to Stein we moved also from using the docker
container engine into Podman. To ensure that every single docker container
was removed after the upgrade a post_upgrade task was added which made
use of the tripleo-docker-rm role that removed the container. In this cycle,
from Stein to Train both the Undercloud and Overcloud work with Podman, so
there is no need to remove any docker container anymore.

This patch removes all the tripleo-docker-rm post-upgrade task and in those
services which only included a single task, the post-upgrade-tasks section
is also erased.

Change-Id: I5c9ab55ec6ff332056a426a76e150ea3c9063c6e
2019-11-12 16:33:38 +01:00
Alex Schultz
f2147c9974 Ensure service log folder permissions
We should ensure that the service folders are 0750. We're setting
/var/log/containers but we should also ensure the service folders also
have the correct permissions.

Change-Id: I28e8017edc7e30a60288adf846da722fd6ab310e
2019-11-04 08:48:24 -07:00
Emilien Macchi
5ff628f1cf Install and start Rsyslog on the Undercloud & Standalone
Create a new Rsyslog service that is deployed on the host (not in a
container) and with Ansible.

Make it so it's deployed by default on Undercloud & Standalone setups.
Also move the tasks that configure rsyslogd for HAproxy & Swift to be
executed after the host prep tasks (using deploy step tasks).

Change-Id: I027c64aefcc4715da17836a5cf0141152cf146aa
Closes-Bug: #1850562
2019-10-31 21:57:01 +01:00
Emilien Macchi
81258ae551 Convert container environment from a list to a dict
Moving all the container environments from lists to dicts, so they can
be consumed later by the podman_container ansible module which uses
dict.

Using a dict is also easier to parse, since it doesn't involve "=" for
each item in the environment to export.

Change-Id: I894f339cdf03bc2a93c588f826f738b0b851a3ad
Depends-On: I98c75e03d78885173d829fa850f35c52c625e6bb
2019-10-16 01:29:31 +00:00
Bogdan Dobrelya (bogdando)
a1e580f039 Revert "Fix generating Apache configs by container-puppet"
fixes following issue coming on RHEL8 http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-master/11c7794/logs/undercloud/var/log/extra/podman/containers/keystone_db_sync/stdout.log.txt.gz

This reverts commit 80d12514d5cd3c20057bd01588e5d5d15d131ca9.

Change-Id: Ice566e90e468bc919872d0954d2d696f4554e00b
2019-08-02 13:54:35 +02:00
Chandan Kumar (raukadah)
c1269a6475 Revert "Wire-in Apache MPM module parameters and switch it"
This reverts commit 09cfcc1464dce0eb7c05caf42375290bbaae4199.

Change-Id: Ife71b124fa404050fcbcb2e041590a295076d6d9
2019-08-02 10:34:07 +00:00
Bogdan Dobrelya
09cfcc1464 Wire-in Apache MPM module parameters and switch it
Allow to configure Apache MPM module for the containerized API/WSGI'ish
services running Apache as a backend. Change the default from 'prefork'
to 'event', which is a low level change and should provide no sensible
upgrade impact. This alleviates the related heartbeats threading issue
arising with the monkey-patched eventlet.

Merge the missing ApacheServiceBase config settings for Octavia API,
Horizon and Ironix PXE. This is needed to apply the base Apache
service hiera settings, including MPM module switches, for those
as well.

Related-bug: #1829062

Change-Id: Ia65af7a9d6ae106a61ec52912bebba72830d5f28
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2019-07-31 10:18:46 +02:00
Bogdan Dobrelya
80d12514d5 Fix generating Apache configs by container-puppet
The changes listed below provide a single unit of work required to
configure Apache backend for WSGI-based OpenStack API services
w/o conflicts causing containers startup failures.

W/o this change /etc/httpd/conf.modules.d/00-mpm.conf shipped with RPM
or other conflicting httpd modules might remain in the containers
and cause startup failures. While puppet removes such conflicts from
the configuration, f.e. when switching MPM 'prefork' to 'event', and we
expect it never gets into container configs.

Make kolla extended start properly enforcing the wanted state of
/etc/httpd, including conf.d and conf.modules.d, and also any of the
removed by puppet files, like conflicting Apache MPM modules.

Add container-puppet tasks to ensure apache MPM configs generated
before the main config steps that require Apache started in the
service container.

Additionally, ensure consistent mirroring across config-data
paths for the container-puppet tool. Purge obsoleted/irrelevant files
in the destingation (puppet-generated) before rsyncing new contents
into it.

Closes-Bug: #1835414

Change-Id: I3e5b4372a01b29bf13179d8a16acc36da9c5caab
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2019-07-31 10:18:30 +02:00
Jose Luis Franco Arza
d1035703b7 Force removal of docker container in tripleo-docker-rm.
The tripleo-docker-rm role has been replaced by tripleo-container-rm [0].
This role will identify the docker engine via the container_cli variable
and perform a deletion of that container. However, these tasks inside the
post_upgrade_tasks section were thought to remove the old docker containers
after upgrading from rocky to stein, in which podman starts to be the
container engine by default.

For that reason, we need to ensure that the container engine in which the
containers are removed is docker, as otherwise we will be removing the
podman container and the deployment steps will fail.

Closes-Bug: #1836531
[0] - 2135446a35

Depends-On: https://review.opendev.org/#/c/671698/
Change-Id: Ib139a1d77f71fc32a49c9878d1b4a6d07564e9dc
2019-07-19 12:37:35 +00:00
Zuul
d61a720177 Merge "Support TLS deployments with KernelDisableIPv6 enabled" 2019-07-10 03:34:33 +00:00
Grzegorz Grasza
d48d1bdb37 Support TLS deployments with KernelDisableIPv6 enabled
Bind to 127.0.0.1 in case ipv6 is disabled. Set a hiera value
localhost_address, so that it can be used in tls_proxy.pp to
unambiguously connect to those services.

Change-Id: Ide761c21dc87dadc722e27c9b8a7b68194164cb2
Related: rhbz#1703460
2019-07-09 16:14:43 +00:00
Emilien Macchi
58bf8a207b swift: ensure we get rsyslog state "--check" mode
Ensure that the logging forwarding tasks do not fail in ansible check
mode, and we make sure the rsyslog_config is actually defined.

Change-Id: Ifaf692643f21f2ae30557b251ae58a9b32fbb143
Closes-Bug: #1835415
2019-07-04 13:55:32 +00:00
Rabi Mishra
90d05216b9 Replace /var/log/containers/swift symlink with directories
Once we remove the symlinks in update_staks, host_prep_tasks
would create those directories.

Assuming that minor update runs before upgrades, this should
take care of upgrades too.

Change-Id: Idabfec5f568a4e02900d103d23f4cb3c199e84c9
Related-Bug: #1833690
2019-06-24 16:18:06 +05:30
Rabi Mishra
d3a9614d8a Don't create symlink for swift container logs
logrotate 3.14.0 silently ignores all files when
there is a broken symlink in the container.

Change-Id: I52a74c7b5473340e2afd240d7d6af9ad5bc97544
Closes-Bug: #1833690
2019-06-21 16:59:21 +05:30
Zuul
14998e6a5d Merge "Convert Docker*Image parameters" 2019-06-18 08:01:14 +00:00
Alan Bishop
06d6fb3bde Remove extraneous references to RpcXXX parameters
Remove references to RpcPort, RpcUserName, RpcPassword and
RpcUseSSL from all templates that do not actually use the parameter.

Change-Id: I295a7ae93feda24a179a53158ecfc633721bcd59
2019-06-14 08:12:32 -04:00
Dan Prince
a68151d02a Convert Docker*Image parameters
This converts all Docker*Image parameter varients into
Container*Image varients.

The commit was autogenerated with the following shell commands:

for file in $(grep -lr Docker.*Image --include \*.yaml --exclude-dir releasenotes); do
  sed -e "s|Docker\([^ ]*Image\)|Container\1|g" -i $file
done

Change-Id: Iab06efa5616975b99aa5772a65b415629f8d7882
Depends-On: I7d62a3424ccb7b01dc101329018ebda896ea8ff3
Depends-On: Ib1dc0c08ce7971a03639acc42b1e738d93a52f98
2019-06-05 14:33:44 -06:00
Dan Prince
a52498ab4d Move containers-common.yaml into deployment
Change-Id: I8cc27cd8ed76a1e124cbb54c938bb86332956ac2
Related-Blueprint: services-yaml-flattening
2019-04-14 18:15:12 -04:00
Andrew Smith
405366fa32 Deprecate messaging params replaced by global oslo params
Depends-On: I03900b39ab257a9563db37e403254b54f846c056
Change-Id: Ib55c72c0bab9aa0ffc05752a680f573cc351ae17
2019-03-28 12:13:07 -06:00