68 Commits

Author SHA1 Message Date
Zuul
26352bb459 Merge "Add missing KOLLA_CONFIG_STRATEGY for the aodh_api_cron container" 2021-04-06 22:32:33 +00:00
Takashi Kajinami
e163846971 Add missing KOLLA_CONFIG_STRATEGY for the aodh_api_cron container
... because kolla_start fails to start with the following error if
that environment parameter is not defined.

ERROR:__main__:InvalidConfig: KOLLA_CONFIG_STRATEGY is not set properly

Change-Id: I7cdf127b495c4d9f415a703fc8b7954a3f5b53fe
2021-04-06 23:51:36 +09:00
ramishra
dba59f9047 Simplify conditions in aodh service templates
Change-Id: I75678ee767871190a598a6e08c9222628ddd90ea
2021-03-31 17:35:15 +05:30
ramishra
c9991c2e31 Use 'wallaby' heat_template_version
With I57047682cfa82ba6ca4affff54fab5216e9ba51c Heat has added
a new template version for wallaby. This would allow us to use
2-argument variant of the ``if`` function that would allow for
 e.g. conditional definition of resource properties and help
cleanup templates. If only two arguments are passed to ``if``
function, the entire enclosing item is removed when the condition
is false.

Change-Id: I25f981b60c6a66b39919adc38c02a051b6c51269
2021-03-31 17:35:12 +05:30
ramishra
b4203a30eb Change all *Debug parameter types to boolean
This changes all these parameters as heat would correctly
parse all values. Also, drops all yaql shenanigans
used for their handling and heat conditions.

Also fixes wrong usage of non-existent NeutronWrapperDebug
parameter in ovn-metadata-container-puppet.yaml.

We had converted all ``Debug`` parameters to boolean with
Ib6c3969d4dd75d5fb2cc274266c060acff8d5571.

Change-Id: Ia2bffffde34aa248a4cc60c3895464f1f9d1ded2
2021-03-30 08:29:10 +05:30
Alexey Stupnikov
8ba48afc6f Remove or fix outdated/incorrect aodh hieradata definitions
- aodh::api::host parameter was removed during Train release.
  Aodh API was for testing purposes only,
  aodh::wsgi::apache::host parameter should be used instead and is
  already defined in THT. Changes:
  I0da4dc1ba52bae5becd5e2a9c0f008cbe2907446 and
  I887d86893e90cebf5bc28dd539e42436f9a31c6a
- there was a typo in aodh::alarm_history_time_to_live definition.

Partial-Bug: #1916386
Change-Id: I8f59623867a1802e2da103b8f56f68dc018f54cb
2021-03-21 15:04:04 +01:00
Grzegorz Grasza
e329ca915e Generate certificates using ansible role
This is using linux-system-roles.certificate ansible role,
which replaces puppet-certmonger for submitting certificate
requests to certmonger. Each service is configured through
it's heat template.

Partial-Implements: blueprint ansible-certmonger
Depends-On: https://review.rdoproject.org/r/31713
Change-Id: Ib868465c20d97c62cbcb214bfc62d949bd6efc62
2021-03-10 16:28:22 +01:00
ramishra
7f195ff9a8 Remove DefaultPasswords interface
This was mainly there as an legacy interface which was
for internal use. Now that we pull the passwords from
the existing environment and don't use it, we can drop
this.

Reduces a number of heat resources.

Change-Id: If83d0f3d72a229d737a45b2fd37507dc11a04649
2021-02-12 11:38:44 +05:30
Michele Baldessari
48d0e4d9b6 Notification drivers need to be a list
Convert the NotificationDriver to a comma_delimited_list.
This will still not break existing templates because passing
a string is still completely valid. This is done so that the hiera keys
will be passed down as lists.

The oslo::messaging::notifications::driver expects a list anyway so this
won't break things and will allow us to actually specify multiple
notification drivers correctly. The change that allowed
oslo::notifications to use both strings and lists is
If65946412b42e0919456ed92fdd8e3788ad67872 (Messaging notifications
should be set as a list)

Related-Bug: #1851629

Change-Id: I24c860cd3121e5c307233864818ca86967ff6d72
2020-12-18 11:26:15 +00:00
Zuul
07302c3b7d Merge "Add aodh-expirer cron job" 2020-11-18 13:04:44 +00:00
Takashi Kajinami
37548ddb40 Enforce internal api for token verification
This change enforces the usage of internal api for token verification,
so that internal requests to keystone uses internal endpoint instead
of admin endpoint which is deployed on provisioning network by default.

Change-Id: I8b5ac36ff1da46844d18fa73f835175e52719a63
Closes-Bug: #1899266
2020-10-11 15:46:08 +09:00
Takashi Kajinami
afc0b731e0 Disable notification from services by default
Currently we disable Telemetry services like Ceilometer by defaut,
which means that we don't have any consumers for notification messages.
So NotificationDriver should be set as noop by default so that we don't
have unconsumed messages in notification queues.

Change-Id: I1d05749c94bd58ad4badafa7d9755009cb4b64af
Closes-Bug: #1869355
2020-09-30 09:51:08 +09:00
Takashi Kajinami
46113c5453 Add aodh-expirer cron job
This patch introduces aodh_api_cron container to aodh-expirer cron job,
which is required to remove old alarm histories from database.

Closes-Bug: #1891888
Depends-on: https://review.opendev.org/#/c/746241/
Depends-on: https://review.opendev.org/#/c/746424/
Change-Id: I023efc0cb6b7775aafa4b1fcc9049197724669e5
2020-08-18 08:16:18 +09:00
Jose Luis Franco Arza
8783ec9c45 Remove ffwd-upgrade leftovers from THT.
Now that the FFU process relies on the upgrade_tasks and deployment
tasts there is no need to keep the old fast_forward_upgrade_tasks.

This patch removes all the fast_forward_upgrade_tasks section from
the services, as well as from the common structures.

Change-Id: I39b8a846145fdc2fb3d0f6853df541c773ee455e
2020-07-23 15:33:25 +00:00
Emilien Macchi
1a48fa61f4 Sync httpd conf.modules.d configs
For containers which run httpd, make sure conf.modules.d is also synced
into the container; so apache doesn't fail with:
AH00534: httpd: Configuration error: More than one MPM loaded.

This is now required since:
6425cc46a8

Change-Id: Ib315d10dbdbbad1628f536a74cd1fca371f018f5
Closes-Bug: #1884115
2020-06-24 03:32:02 +00:00
Zuul
4e5dcf91d4 Merge "Add mode option when creating persistent directories." 2020-05-05 11:41:09 +00:00
Jose Luis Franco Arza
94bc023390 Add mode option when creating persistent directories.
Almost every single tripleo service creates a persistent directory. To
simplify the creation, a with_items structure was being used. In which
many times, the mode option was being set. However, that mode option
was not taken into account at the time of creating the file. As a
consequence, the directory was being created with its father directory
rights, instead of the ones being passed in the template.

Change-Id: I215db2bb79029c19ab8c62a7ae8d93cec50fb8dc
Closes-Bug: #1871231
2020-04-20 15:37:08 +02:00
Takashi Kajinami
dace9fba0c Remove usage of deprecated aodh::auth::auth_tenant_name
... and use aodh::auth::auth_project_name instead.

Depends-on: https://review.opendev.org/#/c/720898/
Change-Id: I3ce473844bbda676204ee3132e46beb33c5320d2
2020-04-20 21:39:17 +09:00
Emilien Macchi
38bad5283f Remove all ignore_errors to avoid confusion when debugging
- deploy-steps-tasks-step-1.yaml: Do not ignore errors when dealing
  with check-mode directories. The file module is resilient enough to
  not fail if the path is already absent.

- deploy-steps-tasks.yaml: Replace ignore_errors by another condition,
  "not ansible_check_mode"; this task is not needed in check mode.

- generate-config-tasks.yaml: Replace ignore_errors by another
  condition, "not ansible_check_mode"; this task is not needed in check mode.

- Neutron wrappers: use fail_key: False instead of ignore_errors: True
  if a key can't be found in /etc/passwd.

- All services with service checks: Replace "ignore_errors: true" by
  "failed_when: false". Since we don't care about whether or not the
  task returns 0, let's just make the task never fail. It will only
  improve UX when scrawling logs; no more failure will be shown for
  these tasks.

- Same as above for cibadmin commands, cluster resources show
  commands and keepalived container restart command; and all other shell
  or command or yum modules uses where we just don't care about their potential
  failures.

- Aodh/Gnocchi: Add pipefail so the task isn't support to fail

- tripleo-packages-baremetal-puppet and undercloud-upgrade: check shell
  rc instead of "succeeded", since the task will always succeed.

Change-Id: I0c44db40e1b9a935e7dde115bb0c9affa15c42bf
2020-03-05 09:22:04 -05:00
Zuul
b130f78076 Merge "Replace svirt_sandbox_file_t by container_file_t" 2020-02-10 13:58:31 +00:00
Zuul
efd47eaec2 Merge "Replace '' by [] when a bind mount isn't needed" 2020-02-08 05:19:17 +00:00
Zuul
c48ccacf74 Merge "Remove deprecated authtoken::auth_uri" 2020-02-07 17:43:51 +00:00
Cédric Jeanneret
0875895553 Replace svirt_sandbox_file_t by container_file_t
While they are, at SELinux level, exactly the same (one is an alias to
the other), the "container_file_t" name is easier to understand (and
shorter to write).

A second pass in a couple of days or weeks will be needed in order to
change files that were merged after this first pass.

Change-Id: Ib4b3e65dbaeb5894403301251866b9817240a9d5
2020-02-07 13:33:20 +01:00
Emilien Macchi
98118b6294 Replace '' by [] when a bind mount isn't needed
To avoid empty volumes like:

{
  (...)
  "volumes": [
    "/etc/puppet:/etc/puppet:ro",
    (...)
    "",
    ""
  ],
}

Replace '' by [], so heat won't create an item in the list.
It helps to have idempotent containers, since podman_container module
will compare the list of volumes that is given in parameters (containing
the empty entries) vs the list of volumes actually in podman inspect.
Replacing to [] clears out empty volumes and makes these containers
idempotent when podman_container module is used to deploy containers.

Change-Id: I228b01009e7d9980bee5480778dbc88b9e226297
2020-02-07 14:34:53 +05:30
Zuul
a3916383d3 Merge "Update ffwd-upgrade branch names" 2020-02-01 21:51:45 +00:00
Zuul
ecce275e47 Merge "Remove stray conditional from aodh-evaluator tasks" 2020-01-29 19:46:52 +00:00
Zuul
a5f1d5c6e2 Merge "Add DeployIdentifier to extra config containers" 2020-01-29 14:44:14 +00:00
Jesse Pretorius (odyssey4me)
2092b1303f Update ffwd-upgrade branch names
The next iteration of fast-forward-upgrade will be
from queens through to train, so we update the names
accordingly.

Change-Id: Ia6d73c33774218b70c1ed7fa9eaad882fde2eefe
2020-01-27 19:42:40 +00:00
Jesse Pretorius (odyssey4me)
edd10576ca Remove stray conditional from aodh-evaluator tasks
In Ib4c59302ad5ad64f23419cd69ee9b2a80333924e a conditional
was moved incorrectly and left as a stray. Luckily it is
only a cosmetic issue, but it might be confusing when doing
any maintenance work so it's better to remove it.

Change-Id: Iac3cbf23437bb11ec47dd1a0f189babe0c5587d0
2020-01-27 19:39:07 +00:00
Takashi Kajinami
8cc62c5f14 Remove deprecated authtoken::auth_uri
auth_uri parameter in authtoken was already removed from puppet modules[1],
so remove it from hieradata.

Also, some service templates missed www_authenticate_uri, which was
introduced as a replacement of auth_uri, so add it to make sure that
we have a correct parameter confugured.

[1] I12b4049e4942911c8d1d8027c579eb4c0d1a53eb

Change-Id: I1e8378f58662377344194916e8bc336df02d0591
2020-01-26 09:26:50 +09:00
Zuul
f739c2134c Merge "Set region in authtoken middleware settings" 2020-01-25 15:45:00 +00:00
Brent Eagles
714e1b5d31 Add DeployIdentifier to extra config containers
Certain config containers might need to be replaced and re-run
regardless of whether configuration changes on update and upgrade.
Adding the DeployIdentifier to the env will ensure that they are.

Change-Id: I150212ebac3fed471ffb4e7ed7b6eb6c7af3fad9
Closes-Bug: #1860571
2020-01-22 15:16:12 -03:30
Kevin Carter
9a2a36437d
Update all roles to use the new role name
Ansible has decided that roles with hypens in them are no longer supported
by not including support for them in collections. This change renames all
the roles we use to the new role name.

Depends-On: Ie899714aca49781ccd240bb259901d76f177d2ae
Change-Id: I4d41b2678a0f340792dd5c601342541ade771c26
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2020-01-20 10:32:23 -06:00
Takashi Kajinami
26305fae91 Set region in authtoken middleware settings
While we can specify keystone region where all keystone resources
are created, currently we don't set the specified region correctly
in credential configurations used for authtoken middleware.

Configure region parameter for authtoken according to the parameter
KeystoneRegion so that we're consistent about the region where
we expect to have service users created.

Change-Id: Icc0ee9a859c2c67cae92339c6b4102946150269f
2020-01-18 21:59:49 +09:00
Emilien Macchi
7f40baabcd Manage all Keystone resources with Ansible
Depends-On: I557d8f33c9c699aed14b3b6fc1d1c0407365cd08
Depends-On: Ia68f8852662fb4abbd194954a246afb740bf3f71

Change-Id: I96a3351fca26cd8bb122a86cb4c3a58d5f88573e
2020-01-06 22:33:05 +00:00
Sagi Shnaidman
016f7c6002 Remove unnecessary slash volume maps
When podman parses such volume map it removes the slash
automatically and shows in inspection volumes w/o slash.
When comparing configurations it turns to be a difference and
it breaks idempotency of containers, causing them to be recreated.

Change-Id: Ifdebecc8c7975b6f5cfefb14b0133be247b7abf0
2019-12-04 20:32:14 +02:00
Kevin Carter
50367fbe35 Convert firewall rules to use TripleO-Ansible
This change converts our filewall deployment practice to use
the tripleo-ansible firewall role. This change creates a new
"firewall_rules" object which is queried using YAQL from the
"FirewallRules" resource.

A new parameter has been added allowing users to input
additional firewall rules as needed. The new parameter is
`ExtraFirewallRules` and will be merged on top of the YAQL
interface.

Depends-On: Ie5d0f51d7efccd112847d3f1edf5fd9cdb1edeed
Change-Id: I1be209a04f599d1d018e730c92f1fc8dd9bf884b
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2019-11-18 15:40:22 -06:00
Jose Luis Franco Arza
4cbae84c75 Get rid of docker removing in post_upgrade tasks.
When upgrading from Rocky to Stein we moved also from using the docker
container engine into Podman. To ensure that every single docker container
was removed after the upgrade a post_upgrade task was added which made
use of the tripleo-docker-rm role that removed the container. In this cycle,
from Stein to Train both the Undercloud and Overcloud work with Podman, so
there is no need to remove any docker container anymore.

This patch removes all the tripleo-docker-rm post-upgrade task and in those
services which only included a single task, the post-upgrade-tasks section
is also erased.

Change-Id: I5c9ab55ec6ff332056a426a76e150ea3c9063c6e
2019-11-12 16:33:38 +01:00
Alex Schultz
7906fb43be Drop legacy log folder and readme
We switched to containers a long time ago. This patch drops the
management of a /var/log/<service> directory and the creation of a
readme indicating that we've moved to containers which makes the logging
available under /var/log/containers/<service>

Change-Id: Ia4e991d5d937031ac3312f639b726a944743dd1e
2019-11-04 09:19:07 -07:00
Alex Schultz
f2147c9974 Ensure service log folder permissions
We should ensure that the service folders are 0750. We're setting
/var/log/containers but we should also ensure the service folders also
have the correct permissions.

Change-Id: I28e8017edc7e30a60288adf846da722fd6ab310e
2019-11-04 08:48:24 -07:00
Zuul
814ca5ed32 Merge "Add SQLAlchemy-collectd support" 2019-10-21 19:38:56 +00:00
Emilien Macchi
81258ae551 Convert container environment from a list to a dict
Moving all the container environments from lists to dicts, so they can
be consumed later by the podman_container ansible module which uses
dict.

Using a dict is also easier to parse, since it doesn't involve "=" for
each item in the environment to export.

Change-Id: I894f339cdf03bc2a93c588f826f738b0b851a3ad
Depends-On: I98c75e03d78885173d829fa850f35c52c625e6bb
2019-10-16 01:29:31 +00:00
Mike Bayer
4bee12fea1 Add SQLAlchemy-collectd support
The SQLAlchemy-collectd plugin is now shipped in podman
containers under Kolla, this allows heat templates
to pull the plugin into the collectd configuration when
the collectd templates are being used.

A corresponding change in puppet-tripleo under the same change-id
adds support to enable the plugin on the puppet side.

The feature can be enabled for an overcloud by adding:

    EnableSQLAlchemyCollectd: true

to the heat configuration while also including one of the
collectd templates from environments/metrics.

The implementation requires that Openstack services which make
use of SQLAlchemy include directives for the plugin within
the SQLAlchemy URL, so this incurs a change in all templates
that include a MySQL database URL.

Change-Id: If598da717653a383a2d3b3373c56517f8bca832f
2019-10-11 10:16:30 -04:00
Carlos Camacho
8529ce60da Stop services for unupgraded controllers
Before we start services on upgraded bootstrap
controller (usually controller-0), we need to
stop services on unupgraded controllers
(usually controller-1 and controller-2).

Also we need to move the mysql data transfer
to the step 2 as we need to first stop the
services.

Depends-On: I4fcc0858cac8f59d797d62f6de18c02e4b1819dc
Change-Id: Ib4af5b4a92b3b516b8e2fc1ae12c8d5abe40327f
2019-08-07 19:23:11 +02:00
Bogdan Dobrelya (bogdando)
a1e580f039 Revert "Fix generating Apache configs by container-puppet"
fixes following issue coming on RHEL8 http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-master/11c7794/logs/undercloud/var/log/extra/podman/containers/keystone_db_sync/stdout.log.txt.gz

This reverts commit 80d12514d5cd3c20057bd01588e5d5d15d131ca9.

Change-Id: Ice566e90e468bc919872d0954d2d696f4554e00b
2019-08-02 13:54:35 +02:00
Chandan Kumar (raukadah)
c1269a6475 Revert "Wire-in Apache MPM module parameters and switch it"
This reverts commit 09cfcc1464dce0eb7c05caf42375290bbaae4199.

Change-Id: Ife71b124fa404050fcbcb2e041590a295076d6d9
2019-08-02 10:34:07 +00:00
Bogdan Dobrelya
09cfcc1464 Wire-in Apache MPM module parameters and switch it
Allow to configure Apache MPM module for the containerized API/WSGI'ish
services running Apache as a backend. Change the default from 'prefork'
to 'event', which is a low level change and should provide no sensible
upgrade impact. This alleviates the related heartbeats threading issue
arising with the monkey-patched eventlet.

Merge the missing ApacheServiceBase config settings for Octavia API,
Horizon and Ironix PXE. This is needed to apply the base Apache
service hiera settings, including MPM module switches, for those
as well.

Related-bug: #1829062

Change-Id: Ia65af7a9d6ae106a61ec52912bebba72830d5f28
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2019-07-31 10:18:46 +02:00
Bogdan Dobrelya
80d12514d5 Fix generating Apache configs by container-puppet
The changes listed below provide a single unit of work required to
configure Apache backend for WSGI-based OpenStack API services
w/o conflicts causing containers startup failures.

W/o this change /etc/httpd/conf.modules.d/00-mpm.conf shipped with RPM
or other conflicting httpd modules might remain in the containers
and cause startup failures. While puppet removes such conflicts from
the configuration, f.e. when switching MPM 'prefork' to 'event', and we
expect it never gets into container configs.

Make kolla extended start properly enforcing the wanted state of
/etc/httpd, including conf.d and conf.modules.d, and also any of the
removed by puppet files, like conflicting Apache MPM modules.

Add container-puppet tasks to ensure apache MPM configs generated
before the main config steps that require Apache started in the
service container.

Additionally, ensure consistent mirroring across config-data
paths for the container-puppet tool. Purge obsoleted/irrelevant files
in the destingation (puppet-generated) before rsyncing new contents
into it.

Closes-Bug: #1835414

Change-Id: I3e5b4372a01b29bf13179d8a16acc36da9c5caab
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2019-07-31 10:18:30 +02:00
Jose Luis Franco Arza
d1035703b7 Force removal of docker container in tripleo-docker-rm.
The tripleo-docker-rm role has been replaced by tripleo-container-rm [0].
This role will identify the docker engine via the container_cli variable
and perform a deletion of that container. However, these tasks inside the
post_upgrade_tasks section were thought to remove the old docker containers
after upgrading from rocky to stein, in which podman starts to be the
container engine by default.

For that reason, we need to ensure that the container engine in which the
containers are removed is docker, as otherwise we will be removing the
podman container and the deployment steps will fail.

Closes-Bug: #1836531
[0] - 2135446a35

Depends-On: https://review.opendev.org/#/c/671698/
Change-Id: Ib139a1d77f71fc32a49c9878d1b4a6d07564e9dc
2019-07-19 12:37:35 +00:00
Zuul
14998e6a5d Merge "Convert Docker*Image parameters" 2019-06-18 08:01:14 +00:00