100 Commits

Author SHA1 Message Date
Alex Schultz
ebab335f38 Role specific container support
We may want to be able to specific different containers at a role level.
This requires switching the container image parameters to be role
specific too allow for role based overrides.

Change-Id: I4090e889a32abd51e7c11139737a7a18e27d18e7
2022-01-21 14:18:02 -07:00
Cédric Jeanneret
7a99ae23e3 Introduce a new linter for yaml-validate, and correct issues
This new linter ensures we don't have any trailing "/" in the container
volume definitions.

Those trailing "/" may create issues with the containers, for instance
for specific mounts such as "/dev"[1].

This patch also takes the opportunity to fix those trailing "/" for the
affected files, in order to start on a clean basis.

[1] https://launchpad.net/bugs/1950176

Change-Id: If951f9643d67574c1225301aab7c9e4b0d316b7f
Related-Bug: #1950176
2021-12-01 09:43:25 +01:00
Matthias Runge
c25eec5bff Limit collectds memory usage in the ansible part
In analogy to I0b65b2c7878fe5ebd0d7f1a6f2ba8df23f2f0b28
where the memory was limited in the puppet part

Change-Id: I83a245b51dd58457001e82e0ee5b04c1fc6de2ac
2021-10-04 20:38:02 +02:00
Martin Magr
e8beaaf880 Add logging source for collectd
Change-Id: I065e67f962ea17f10ce45aaa0497a7bf61d74d28
2021-09-09 22:30:18 +02:00
Alex Schultz
a94ff466c8 Example configurable cap add
Change-Id: Ib34130c15324f197eca275f97c18f24f17d8ced2
2021-07-28 14:47:16 +02:00
Zuul
b6173acb8f Merge "Clean up DefaultPasswors" 2021-07-02 12:51:40 +00:00
Zuul
4c82552bc6 Merge "Limit collectds memory usage" 2021-06-21 09:45:20 +00:00
Matthias Runge
2f702f7bb1 Limit collectds memory usage
to guard nodes from running out of memory.

If there is a connection issue with collectd not being able to
write to event or metrics endpoints, this can lead to accumulating
to large portions of memory. Limiting the memory usage via podman
memory will prevent the container to grow limitless. In case of
using more than 512 MB, podman will restart the container.

Change-Id: I0b65b2c7878fe5ebd0d7f1a6f2ba8df23f2f0b28
2021-06-11 16:42:16 +02:00
Takashi Kajinami
f86a22d338 Clean up DefaultPasswors
DefaultPasswords is supposed to have been removed[1] but there still
exist some leftovers. This change removes all of the remaining usage.

[1] 7f195ff9a81c97469cd3c74d441c28c3b07d34ba

Change-Id: I4cd0076378c3e6b99cec62b0794ce68787256a1b
2021-06-08 22:56:07 +09:00
Zuul
31eba7e9eb Merge "Pass scripts list for download for sensubility" 2021-06-07 20:34:47 +00:00
Martin Magr
66c964bf91 Pass scripts list for download for sensubility
Depends-On: Ifb1046d710827a82dcecbfbb2a449b1f87bbe363
Change-Id: I95ab281b6f4f9402feae26487b922e6a59884a99
2021-05-20 14:36:47 +02:00
ramishra
b253d564f7 Use server side env merging for ServiceNetMap/VipSubnetMap
This simplifies the ServiceNetMap/VipSubnetMap interfaces
to use parameter merge strategy and removes the *Defaults
interfaces.

Change-Id: Ic73628a596e9051b5c02435b712643f9ef7425e3
2021-05-19 10:16:58 +05:30
Zuul
c737ed6655 Merge "Simplify metrics and messaging service templates" 2021-05-04 23:06:38 +00:00
ramishra
c5c558229e Simplify metrics and messaging service templates
Change-Id: Ibd240fee7df197301ef17c8029d44d06d6007bc5
2021-04-28 09:42:18 +05:30
pleimer
cda21df470 Correct metrics_qdr logging path and regex parsing
Change-Id: I98c1b7b21f9d5bd6e0f91ed7b1df74d01ab2b592
Resolves: rhbz#1945075
2021-04-23 18:35:00 +00:00
Chris Sibbitt
1a95607b82 Removing duplicate mount point in metrics_qdr
The metrics_qdr container fails to start when tls-everywhere is enabled
due to a duplicate mount point for /etc/ipa/ca.crt already configured in
containers-common[1]

[1] https://review.opendev.org/plugins/gitiles/openstack/tripleo-heat-templates/+/refs/heads/master/deployment/containers-common.yaml#146

Change-Id: I8acd10b2d5624066af3a9426081f9a0679890b3e
Resolves: rhbz#1947116
2021-04-15 14:30:21 -04:00
Zuul
824ec8b5ad Merge "Simplify internal_tls_enabled conditions" 2021-04-03 13:20:28 +00:00
ramishra
c9991c2e31 Use 'wallaby' heat_template_version
With I57047682cfa82ba6ca4affff54fab5216e9ba51c Heat has added
a new template version for wallaby. This would allow us to use
2-argument variant of the ``if`` function that would allow for
 e.g. conditional definition of resource properties and help
cleanup templates. If only two arguments are passed to ``if``
function, the entire enclosing item is removed when the condition
is false.

Change-Id: I25f981b60c6a66b39919adc38c02a051b6c51269
2021-03-31 17:35:12 +05:30
Zuul
6edb6760f1 Merge "Provide ability to deploy metrics_qdr using ansible" 2021-03-31 09:54:02 +00:00
Michele Baldessari
5e4c17acfb Simplify internal_tls_enabled conditions
We do not need to add an if: internal_tls_enabled in a number of
ansible tasks. enabled_internal_tls is already defined as an ansible
fact in common/deploy-steps.j2:
enable_internal_tls: {get_param: EnableInternalTLS}

So when the service uses the enable_internal_tls condition and it points
to the EnableInternalTLS param, we can just use the ansible fact
directly. Note that if the enable_internal_tls condition points to
something else than the mere EnableInternalTLS we may not do this
cleanup.

Change-Id: Idb07cbc8fc3a4d73ff52c54d869310fd6c49b502
2021-03-27 13:42:35 +01:00
Zuul
bb81090ef1 Merge "[collectd][ansible] Add THT to deploy collectd using ansible" 2021-03-27 08:49:17 +00:00
Chris Sibbitt
5c21f8df6d Provide ability to deploy metrics_qdr using ansible
Change-Id: I75154fe9b61b4ac655ce06b3b5bbcbe31e1381fb
2021-03-24 08:37:16 -04:00
Emma Foley
4ddc178cdc [collectd][ansible] Add THT to deploy collectd using ansible
All heat params have been copied over, there are a bunch
that are used for conditionals.
The outputs and conditionals secions in *-puppet do a lot
of configuration, and provides lists of defaults for
puppet. These will be moved to ansible, role is at [1]
and in tripleo_ansible.

[1] https://github.com/infrawatch/collectd-config-ansible-role
[x] https://github.com/infrawatch/tripleo-collectd-ansible-role

Depends-On: Ib75702bf17a76cae3a811db503d3365e6aacf663
Change-Id: I9939a524795bb3fbc63e44f203f851dadeb7c30a
2021-03-22 15:35:08 +00:00
Carlos Goncalves
6e7e0ab48e Remove obsoleted generate_service_certificates
Remove traces of generate_service_certificates. It was removed during
Pike release cycle [1].

[1] https://review.opendev.org/c/openstack/puppet-tripleo/+/444891

Change-Id: Ib203b52547433ff73141df66641528c389b50361
2021-03-16 19:50:14 +01:00
Grzegorz Grasza
e329ca915e Generate certificates using ansible role
This is using linux-system-roles.certificate ansible role,
which replaces puppet-certmonger for submitting certificate
requests to certmonger. Each service is configured through
it's heat template.

Partial-Implements: blueprint ansible-certmonger
Depends-On: https://review.rdoproject.org/r/31713
Change-Id: Ib868465c20d97c62cbcb214bfc62d949bd6efc62
2021-03-10 16:28:22 +01:00
ramishra
7f195ff9a8 Remove DefaultPasswords interface
This was mainly there as an legacy interface which was
for internal use. Now that we pull the passwords from
the existing environment and don't use it, we can drop
this.

Reduces a number of heat resources.

Change-Id: If83d0f3d72a229d737a45b2fd37507dc11a04649
2021-02-12 11:38:44 +05:30
Jesse Pretorius (odyssey4me)
46df551a0f Use include_role for conditional inclusion
When import_role is used with a condition, the condition is
applied to all tasks in the role. This is inefficient. If we
use include_role instead, then the role inclusion task is
skipped and none of the tasks in the role are even evaluated.

Related-bug: rhbz#1922132

Change-Id: I1b4e26fd4a46599985a989441f493a3ed39237bb
2021-01-29 10:39:16 +00:00
Raildo
9760977529 Adding key_size option on the certificate creation
Adding the ability to specifies the private key size
used when creating the certificate. We have defined the
default value the same as we have before 2048 bits.
Also, it'll be able to override the key_size value
per service.

Depends-on: I4da96f2164cf1d136f9471f1d6251bdd8cfd2d0b
Change-Id: Ic2edabb7f1bd0caf4a5550d03f60fab7c8354d65
2020-12-17 20:22:52 -03:00
Zuul
47feac29c5 Merge "Fix MetricsQdrUseSSL value" 2020-11-03 14:11:05 +00:00
Zuul
63d0cd4a03 Merge "Add SendQueueLimit to collectd-amqp1" 2020-11-03 09:19:03 +00:00
Martin Magr
dfc3322b63 Fix MetricsQdrUseSSL value
puppet-qdr expects one of the string values 'yes'/'no' instead of boolean,
so currently listeners cannot be created with SSL enabled. This patch is fixing
this issue.

Change-Id: I2dbafdc417f31eb000fbb469ae2c082cda3a5523
2020-10-30 14:44:20 +01:00
pleimer
6fef2d86b2 Config options for AMQP1 transport in collectd sensubility
This patch exposes configuration options for setting up collectd
sensubility to use AMQP1 transport. This is primarily for
compatability with the Service Telemetry Framework.

Change-Id: Id74eb494e022e88f0a1d5c1044d3a7a283b5178f
Depends-On: https://review.opendev.org/#/c/758482/
2020-10-19 16:52:47 -04:00
Matthias Runge
91cc43e01f Add SendQueueLimit to collectd-amqp1
to limit the queue length and to ignore data added
earlier to the queue, when there is no remote to
connect to. This will limit the memory usage of
this plugin. By default, the memory may grow until
it runs out of memory.

Depends-On: https://review.opendev.org/756365
Change-Id: Ie2db4b7487f9a8eaa7152b1b7984c52016219ed5
2020-10-12 13:41:56 +00:00
pleimer
824f1b8d3b Add config option for collectd libpodstats
Change-Id: I1f47772d01e10b3f399bd4ba3da90b6016e407b0
Depends-On: https://review.opendev.org/754838
Signed-off-by: pleimer <pfbleimer@gmail.com>
2020-09-29 14:08:41 -04:00
Zuul
0d8ecc4593 Merge "Gather more extra stats with the collectd virt plugin" 2020-09-24 00:39:01 +00:00
Zuul
be04d1536a Merge "Adapt container health check for built-in podman health checks" 2020-09-23 03:56:56 +00:00
Zuul
7c8108cced Merge "Centralized logging minor fixes" 2020-09-17 22:24:32 +00:00
Paul Leimer
fabe5b3b70 Gather more extra stats with the collectd virt plugin
Change-Id: Ida1b12267899b8604770b16f0feb8165a2010359
2020-09-15 23:53:30 +00:00
Martin Magr
1952a9ce64 Adapt container health check for built-in podman health checks
This patch removes regression which was introduced by moving from systemd
health check framework to built-in podman health check support.

Change-Id: I1706e04b543e8c9ff3903a9575b7c2cd74b9a0b3
2020-09-15 16:52:56 +02:00
Martin Magr
34a2b091d2 Centralized logging minor fixes
This patch contains following fixes to have rsyslog configured properly:

 - Don't use default startmsg.regex for QDR
 - Fixed log path for pacemaker service

Change-Id: I2220ba108296475a49eebdc3ceec27208d201b0d
2020-09-08 15:05:32 +02:00
pleimer
fdf8738614 Mount libpod container volume into collectd container
The collectd-libpod-stats plugin requires additional libpod volumes to
be mounted into the collectd contianer in order to find and track running
containers. This mounts the only additional volume necessary

Change-Id: I0f3fb05d8295f8707ad041debb250f255d20626f
Signed-off-by: pleimer <pfbleimer@gmail.com>
2020-09-02 10:25:15 -04:00
Jose Luis Franco Arza
8783ec9c45 Remove ffwd-upgrade leftovers from THT.
Now that the FFU process relies on the upgrade_tasks and deployment
tasts there is no need to keep the old fast_forward_upgrade_tasks.

This patch removes all the fast_forward_upgrade_tasks section from
the services, as well as from the common structures.

Change-Id: I39b8a846145fdc2fb3d0f6853df541c773ee455e
2020-07-23 15:33:25 +00:00
Cédric Jeanneret
e047d83142 Ensure we use only /run instead of /var/run
There is no real value using /var/run instead of /run, especially since
/var/run is a symlink to /run.

This patch also removes duplicated mounts due to this very symlink.

Change-Id: Iaced2ba676a4e4f651c67da082797cc1c1ffccd1
2020-07-06 07:19:02 +00:00
Zuul
1dfb47816f Merge "Unify metrics_qdr name to underscore" 2020-06-09 05:25:20 +00:00
michalrebisz
a0cbe76723 collectd: add support for mcelog service
Adding a new parameter, CollectdEnableMcelog (False by default) is set
to True, a new host prep task will include the new
"tripleo_provision_mcelog" role to configure the mcelog service on the
host where the collectd container is running.

Depends-On: https://review.opendev.org/730409

Change-Id: Ia2280092abb0d769a92994ad337ed2b583f54175
2020-05-25 08:53:23 +00:00
Martin Magr
8dbaed0c31 Unify metrics_qdr name to underscore
Some parameter values of the MetricsQdr service used "metrics-qdr"
and some "metrics-qdr". This patch unifies to the one with underscore.

Change-Id: I9767953383a82f8baa50289e88ba279798bbfe92
2020-05-07 11:42:48 +02:00
Zuul
37a2f8b8cb Merge "Change Collectd ports type to numbers." 2020-05-06 10:43:16 +00:00
Jose Luis Franco Arza
e0bf4579e4 Change Collectd ports type to numbers.
In [0] it was changed the type of some of the ports
used in the service. The template needs to be adapted
to be able to work correctly.

[0] - https://github.com/voxpupuli/puppet-collectd/commit/d7b79c

Change-Id: Ied613222a593566634c69fd63f891f0ac1497f86
Related-Bug: #1871086
2020-04-21 12:02:27 +00:00
Jose Luis Franco Arza
94bc023390 Add mode option when creating persistent directories.
Almost every single tripleo service creates a persistent directory. To
simplify the creation, a with_items structure was being used. In which
many times, the mode option was being set. However, that mode option
was not taken into account at the time of creating the file. As a
consequence, the directory was being created with its father directory
rights, instead of the ones being passed in the template.

Change-Id: I215db2bb79029c19ab8c62a7ae8d93cec50fb8dc
Closes-Bug: #1871231
2020-04-20 15:37:08 +02:00
Takashi Kajinami
fffdcf0f30 Use absolute name to include puppet classes
Current puppet modules uses only absolute name to include classes,
so replace relative name by absolute name in template files so that
template description can be consistent with puppet implementation.

Change-Id: I7a704d113289d61ed05f7a31d65caf2908a7994a
2020-04-11 08:13:23 +09:00