Since https://review.openstack.org/#/c/514707/ added the net_ip_map
to hieradata, we can look up the per-network bind IPs via hiera
interpolation instead of heat map_replace.
In some cases the ServiceNetMap lookup is used for other things,
but anywhere we make use of the "magic" translation via NetIpMap
is changed the same way.
This will enable more of the configuration data to be exposed per
role vs per node in a future patch (to simplify our ansible
workflow).
Co-authored-by: Bogdan Dobrelya <bdobreli@redhat.com>
Change-Id: Ie3da9fedbfce87e85f74d8780e7ad1ceadda79c8
If we use variables defined in later step in conditional before
checking which step are we on we will fail.
Resolves: rhbz#1535457
Closes-Bug: #1743764
Change-Id: Ic21f6eb5c4101f230fa894cd0829a11e2f0ef39b
The initial fluentd client implementation predates the introduction of
service_config_settings, and necessitated some invasive changes to
what is now common/serivces.yaml. This commit modifies existing
services to use the service_config_settings based configuration
mechanism supported by more recent versions of the fluentd support in
puppet-tripleo.
Partial-bug: #1715187
Depends-On: I3149902401d68d6fd236073a73a20f982d4b952a
Depends-On: I2b057190ec0e4e75ee4ee47ebe0164c2644e5ab7
Depends-On: Ie7df4b8b94cb0ae38096ab95800f211ef1cd8455
Change-Id: I28028ffa00df2da8e0478a551d3de89c3ee46e1f
This patch reverts the revert of Redis TLS [1,2], and update the
pacemaker redis template to configure Redis to encrypt the
replication traffic between Redis nodes.
[1] a3769c03175cb36f0066c173477749a26f767566
[2] ebc8414cd0c18426ff80d9d65c964e91a7fe447f
Depends-On: I6cc818973fab25b4cd6f7a0d040aaa05a35c5bb1
Change-Id: I7f7be4bba6d41c04385f074857c82507cc8c2617
Closes-Bug: #1737707
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.
This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)
The yaml-validate also now checks for duplicate 'when:' statements
Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
Lets revert the tls support until we know it works.
Revert "TLS proxy for redis"
This reverts commit c2a93cf4c5d9d6b5ee0536380751a7a9540927cc.
Closes-bug: #1735259
Change-Id: I8157ce04617c094978175f3e4b3071bdf76362fe
Because puppet-mongodb requires it to be an array and not a string.
See 23068c10a4
Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Change-Id: I3c1ea4f350ecaea2efcf53ebf8ac26e4936054f7
Closes-Bug: #1726217
When deploying on IPv6 networks, set a hiera key that can be consumed
by puppet modules to configure MySQL or Galera appropriately.
Currently, this is required for configuring SST encryption in Galera
when Internal TLS is enabled [1].
[1] I1d6ee8febb596b3ab9dcde3a85a028ee99b2798c
Change-Id: Ia857350ac451fc1bda6659d85019962d3a9d5617
Partial-Bug: #1719885
Redis does not have TLS out of the box. Let's use a proxy container for
TLS termination.
bp tls-via-certmonger
Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Change-Id: Ie2ae0d048a71e1b1b4edb10c74bc0395a1a9d5c9
Depends-On: I078567c831ade540cf704f81564e2b7654c85c0b
Depends-On: Ia50933da9e59268b17f56db34d01dcc6b6c38147
We install redis if its not already there, but we should also
ensure redis service is started in the next step 4.
related to issues we're seeing in I284de61bbefac9e9b37390650016643ffe38b5cc
Change-Id: Ic01db53ea8669f14e87f6987045b2be5a3480024
Since these are obviously global parameters they shouldn't specify
what will be using them because they are used in multiple places.
Change-Id: I5054c2d67dffe802e37f8391dd7bad4721e29831
Partial-Bug: 1700664
https://github.com/camptocamp/puppet-systemd/pull/32 is disabling by default the services so we don't have to control them via TripleO.
This reverts commit d24874c7b2625e25630534a86864a93050f661d3.
Change-Id: I4044f0b28b636c7a022912f6f24707bce22c8b98
Related-Bug: #1704160
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.
Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).
Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
Latest commits in puppet-systemd enabled by default systemd-networkd and
systemd-resolved but we don't want to manage them for now in TripleO.
MySQL and MongoDB services were managing some systemd resources so now
we ensure that these 2 systemd services are disabled. In the future, we
might want and activate these services and revert that patch but for now
we want to disable them.
Change-Id: I42c6c9b643a71a0fbb1768bbae91e8bfa916ea00
Closes-Bug: #1704145
This will add the node's FQDN to the mysql certificate request
besides the VIP's FQDN which we already use. This is needed for
adding TLS to the replication traffic. The CA file was also added
as hieradata, since the path will be needed for the TLS
configuration.
bp tls-via-certmonger
Change-Id: I9252303b92a2805ba83f86a85770db2551a014d3
This configures the mongodb server to use TLS in the internal network,
while also passing the necessary attributes to generate the needed cert
and key.
bp tls-via-certmonger
Depends-On: I85dda29bcad686372a74bd7f094bfd62777a3032
Change-Id: If6c603b074cfa7e122579cec29d034fd3312868d
When a service is enabled on multiple roles, the parameters for the
service will be global. This change enables an option to provide
role specific parameter to services and other templates.
Two new parameters - RoleName and RoleParameters, are added to the
service template. RoleName provides the role name of on which the
current instance of the service is being applied on. RoleParameters
provides the list of parameters which are configured specific to the
role in the environment file, like below:
parameters_default:
# Default value for applied to all roles
NovaReservedHostMemory: 2048
ComputeDpdkParameters:
# Applied only to ComputeDpdk role
NovaReservedHostMemory: 4096
In above sample, the cluster contains 2 roles - Compute, ComputeDpdk.
The values of ComputeDpdkParameters will be passed on to the templates
as RoleParameters while creating the stack for ComputeDpdk role. The
parameter which supports role specific configuration, should find the
parameter first in in the RoleParameters list, if not found, then the
default (for all roles) should be used.
Implements: blueprint tripleo-derive-parameters
Change-Id: I72376a803ec6b2ed93903cc0c95a6ffce718b6dc
Via https://github.com/arioch/puppet-redis/pull/192 puppet-redis grew
ulimit support also for pacemaker managed redis instances. To be able to
use that we need to set redis::managed_by_cluster_manager to true.
We also allow redis::ulimit to be configurable and we set a default of
10420 which was the default value before the above change.
Change-Id: I06129870665d7d3bfa09057fd9f0a33a99f98397
Depends-On: I4ffccfe3e3ba862d445476c14c8f2cb267fa108d
Closes-Bug: #1688464
Instead of using the CA bundle, this sets the mysql client configuration
file to use a specific file for validating the certificate of the
database server. This helps in two ways:
* Improves performance since validation will check only one certificate.
* Improves security since we're only the certificates signed by one CA
are valid, instead of any certificate that the system trusts (which
could include potentially compromised public certs).
Change-Id: I46f7cb6da73715f8f331337e0161418450d5afd7
Depends-On: I75bdaf71d88d169e64687a180cb13c1f63418a0f
The puppet-tripleo change was added in
Ie9391aa39532507c5de8dd668a70d5b66e17c891.
Closes-bug: #1656558
Change-Id: Ibe2e4be5b5dc953d8d4b14f680a460409db95585
Usually a nested stack is used that contains the TLS-everywhere bits
(config_settings and metadata_settings). Nested stacks are very
resource intensive. So, instead of doing using nested stacks, this patch
changes that to use a conditional, and output the necessary
config_settings and metadata_settings this way in an attempt to save
resources.
Change-Id: Ib7151d67982957369f7c139a3b01274a1a746c4a
We currently do not bind redis-sentinel to any IP:
redis 21144 0.0 0.0 142908 5908 ? Ssl 07:43 0:11 /usr/bin/redis-sentinel *:26379 [sentinel]
Let's bind it to the same network as redis.
Change-Id: I8a782ae1db84eb614aa3995a1638a2f370e70d06
Partial-Bug: #1673715
There is currently an issue where the max open files limit is hit with
MariaDB in noha deployments, because it is defaulted to 1024 by system
limits. In HA deployments the limit is bumped to 16384. This patch
introduces a flag to be able to increase the limit to 16384 for noHA
deployments.
In the future we should change this to be an integer, and let the
operator decide the setting. Since this setting is set in a different
path for HA, we would need to implement a change that allows setting
both (ha and nonha) via the same integer param.
Depends-On: Ia0907b2ab6062a93fb9363e39c86535a490fbaf6
Closes-Bug: #1648181
Related-Bug: #1524809
Change-Id: I95393fc798b833a8575afbff03ef74a839565c5e
Signed-off-by: Tim Rozet <trozet@redhat.com>
It was using a hiera key, and fetching that from a hiera call in the
puppet manfiest. But we can remove that if we set it via hiera from
t-h-t.
Change-Id: I5af5ccb88e644f4dd25503d8e7a93796695d3039
This uses the mysql client configuration file to configure if SSL should
be used for the connection if SSL in the internal network is enabled.
Change-Id: Ifd1a06e0749a05a65f6314255843f572d2209067
When fixing LP#1643487 we added ?bind_address to all DB URIs.
Since this clashes with Cellsv2 due to the URIs becoming host
dependent, we need a new approach to pass bind_address to pymysql
that leaves the DB URIs host-independent.
In change Iff8bd2d9ee85f7bb1445aa2e1b3cfbff1f397b18 we first create a
/etc/my.cnf.d/tripleo.cnf file with a [tripleo] section with the correct
bind-address option.
In this change we make sure that the DB URIs will point to the added
file and to the specific section containing the necessary bind-address
option. We do introduce a new MySQLClient profile which will hold all
this more client-specific configuration so that this change can fit
better in the composable roles work. Also, in the future it might
contain the necessary configuration for SSL for example.
Note that in case the /etc/my.cnf.d/tripleo.cnf file does not exist
(because it is created via the mysqlclient profile), things keep on
working as usual and the bind-address option simply won't be set, which
has no impact on hosts where there are no VIPs.
Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
Change-Id: Ieac33efe38f32e949fd89545eb1cd8e0fe114a12
Related-Bug: #1643487
Closes-Bug: #1663181
Closes-Bug: #1664524
Depends-On: Iff8bd2d9ee85f7bb1445aa2e1b3cfbff1f397b18
These are only used for TLS-everywhere, and fills up the kerberos
principals that will need to be created for the certs used by the
overcloud. With this, the metadata hook will format these principals
correctly and will further pass them on to the nova metadata service.
Where they can be used if there's a plugin enabled.
bp tls-via-certmonger
bp novajoin
Change-Id: I873094bb69200052febda629fda698a7a782c031
Adds a step0 for any pre-upgrade checks. This migrates
some of the checks we have at the top of
extraconfig/tasks/major_upgrade_controller_pacemaker_1.sh
Checks for other services (and for the cluster) will follow
in separate commits.
Partially-Implements: blueprint overcloud-upgrades-per-service
Change-Id: I607f1fed68d7f11773484c3d7cb3e5af67465d57
When a service connects to the database VIP from the node hosting this
VIP, the resulting TCP socket has a src address which is by default
bound to the VIP as well. If the VIP is failed over to another node
while the socket's Send-Q is not empty, TCP keepalive won't engage and
the service will become unavailable for a very long time (by default
more than 10m).
To prevent failover issues, DB connections should have the src address
of their TCP socket bound to the IP of the network interface used for
MySQL traffic. This is achieved by passing a new option to the
database connection URIs. This option is available starting from
PyMySQL 0.7.9-2.
We use a new intermediate variable in hiera to hold the IP to be used
as a source address for all DB connections. All services adapt their
database URI accordingly.
Moreover, a new YAML validation check is added to guarantee that new
services will construct their database URI appropriately.
Change-Id: Ic69de63acbfb992314ea30a3a9b17c0b5341c035
Closes-Bug: #1643487
Heat now supports release name aliases, so we can replace
the inconsistent mix of date related versions with one consistent
version that aligns with the supported version of heat for this
t-h-t branch.
This should also help new users who sometimes copy/paste old templates
and discover intrinsic functions in the t-h-t docs don't work because
their template version is too old.
Change-Id: Ib415e7290fea27447460baa280291492df197e54
This shows how we could wire in the upgrade steps using Ansible
as was previously proposed e.g in https://review.openstack.org/#/c/321416/
but it's more closely integrated with the new composable services
architecture.
It's also very similar to the approach taken by SpinalStack where
ansible snippets per-service were combined then run in a series of
steps using Ansible tags.
This patch just enables upgrade of keystone - we'll add support for
other patches in subsequent patches.
Partially-Implements: blueprint overcloud-upgrades-per-service
Change-Id: I39f5426cb9da0b40bec4a7a3a4a353f69319bdf9
This changes how we get the network-based FQDNs for the specific
services, from using the custom fact, to the new hiera entries.
Change-Id: Iae668a5d89fb7bee091db4a761aa6c91d369b276
This patch optimizes how we deploy hiera by using a new
heat hook specifically designed to help compose hiera
within heat templates. As part of this change:
- we update all the 'hiera' software configurations to set the group to hiera
instead of os-apply-config.
- The new format uses JSON instead of YAML. The hook actually writes
out the hiera JSON directly so no conversion takes place. Arrays,
Strings, Booleans all stay in their native formats. As such we can avoid
having to do many of the awkward string and list conversions in t-h-t to
support the previous YAML formatting.
- The new hook prefers JSON over YAML so upgrading users will have the
new files prefered. (we will post a cleanup routine for the old files
soon but this isn't a new behavior, JSON is now simply prefered.)
- A lot of services required edits to account for default settings that
worked in YAML that no longer work correctly in the native JSON
format. In almost all these cases I think the resulting codes looks
cleaner and is more explicit with regards to what is getting
configured in hiera on the actual nodes.
Depends-On: I6a383b1ad4ec29458569763bd3f56fd3f2bd726b
Closes-bug: #1596373
Change-Id: Ibe7e2044e200e2c947223286fdf4fd5bcf98c2e1
