250 Commits

Author SHA1 Message Date
Martin Schuppert
20dbe32060 Remove NovaConsoleauth Service
As of Rocky [1], the nova-consoleauth service has been deprecated and
cell databases are used for storing token authorizations. All new consoles
will be supported by the database backend and existing consoles will be
reset. Console proxies must be run per cell because the new console token
authorizations are stored in cell databases.

nova-consoleauth was deprecated in tripleo with:
I68485a6c4da4476d07ec0ab5e7b5a4c528820a4f

This change now removes the NovaConsoleauth Service.

[1] https://docs.openstack.org/releasenotes/nova/rocky.html

Closes-Bug: #1828414

Change-Id: Icdfbf26b5e83cc07a560eb227a0cf822e4c5a1e3
2019-05-09 15:24:13 +02:00
Zuul
694777ba05 Merge "Introduce scale_tasks" 2019-04-30 23:54:00 +00:00
Lee Yarwood
63c4518693 nova: Remove the NovaPlacement service
Deprecated by Ic904aba7c19eda0a0a22dd2d13b9d4182b4595c4 the service can
now be removed in Train.

Change-Id: Ic1d59f93f930f093164ac294730a5cbb08a5bded
2019-04-26 17:33:09 +02:00
Emilien Macchi
5d66b9c1eb Introduce scale_tasks
Composable service templates can now define scale_tasks. They are
meant for scale down/up logic of services which need to be
stopped/started during the scaling procedure. All happens within a single
playbook and the down/up Ansible tags are required to differenciate them
during the run.

blueprint scale-down-tasks

Change-Id: I29835c2724b5de884e9825e084780c36837bff84
Co-Authored-By: Alex Schultz <aschultz@redhat.com>
2019-04-24 11:07:53 -04:00
Lee Yarwood
555178160b placement: Introduce an extracted PlacementAPI service
This change introduces an optional extracted version of the Placement
service into TripleO. This extracted version will only be required once
the Placement service is fully removed from Nova during the T cycle
(previously S but delayed) at which point the corresponding
NovaPlacement service will also be removed from TripleO.

The majority of this change is code motion between the original
NovaPlacement service and the new PlacementAPI service.

Upgrades from the original NovaPlacement service to the extracted
PlacementAPI service are not currently supported by this change and will
be worked on independently during the Train cycle.

Co-authored-by: mschuppert@redhat.com

Depends-On: https://review.openstack.org/#/c/624335/
Change-Id: I9e3287bcbe9d317f32bf6b468c6ee17f04b6fff9
2019-04-18 17:34:06 +02:00
Harald Jensås
d5ecc1f651 Make krb-service-principal metadata per-Role
Not all roles are connected to all networks, there is no
need to create metadata for networks not associated with
the role.

In edge/spine-and-leaf deployments the total number of
composable networks used can be high. Passing all the
networks we quickly go beyond the nova metadata fields
size limit (each field cannot exceed 256 bytes).

Also update tools/check-up-to-date.sh script to use the
simple yaml-diff.py instead of diff. The env generator
code will sort data, while jinja rendered environments
are not sorted, thus need to diff the data in yaml not
the text.

Closes-Bug: #1821377
Change-Id: I5ae3bc845b0a6ad6986d44b14ff4b0737a9b033b
2019-04-05 14:22:20 +00:00
Zuul
e52cdf2fc8 Merge "Enable flat network for ovn" 2019-03-30 00:04:59 +00:00
Zuul
bb89de950f Merge "Add external_resource_vip_id property to network_data.yaml" 2019-03-26 03:46:56 +00:00
James Slagle
c7a97ce997 Add external_resource_vip_id property to network_data.yaml
Adds the external_resource_vip_id property, which can be used to set an
external_id for the port resource for the network VIP.

Since the same template resource, port.network.j2.yaml is used for both
VIP and normal ports on a network, we can't simply add jinja to that
template that conditionally adds the external_id attribute because we
don't know during the jinja2 phase if the template is for a VIP or not.

Instead, we need to map the VIP resources to an entirely new template
resource (external_resource_port.network.j2.yaml) so that we can set the
external_id attribute just for the VIP ports.

Change-Id: I27d3eeb11277004b00aa4d6a66014d5c71081c26
implements: blueprint split-controlplane-templates
2019-03-25 10:48:40 -04:00
Kamil Sambor
98c2b78dcb Enable flat network for ovn
Change-Id: I0883b4f1881887cae68da9658d2e237218b694c4
Closes-Bug: #1821171
2019-03-22 09:00:16 +01:00
Dan Prince
5c8456c5cd move ceph-ansible into deployment
Change-Id: I67dc1c2dd7d1e0308db660577f25d74a6305dae6
Related-Blueprint: services-yaml-flattening
2019-03-22 01:01:30 +00:00
Emilien Macchi
160cddda3f Rename docker_config_scripts to container_config_scripts
Change-Id: Iabd65560c2fc28b3aeca07a21efa861c4c583c01
2019-03-06 09:05:50 -05:00
Emilien Macchi
3e62d483fc Rename docker_puppet_tasks to container_puppet_tasks
Change-Id: I23c539555fe1f9db96cd9b84fdcb4240473d55df
2019-03-05 12:07:09 -05:00
Dan Prince
ebc9dd98e0 flatten the octavia service configurations
This change combines the previous puppet and docker files into a single
file that performs the containerized service installation and configuration
for the octavia services.

With this patch the baremetal version of each respective octavia service
has been removed.

Related-Blueprint: services-yaml-flattening

Change-Id: Icf2856fd261b49a4da1f197c7190c9a18d21e30f
2019-03-04 15:28:21 -05:00
Zuul
ef859797e4 Merge "Fix py3 compat for regex searches in YAML validate" 2019-02-28 13:21:18 +00:00
Bogdan Dobrelya
d2eba382cc Fix py3 compat for regex searches in YAML validate
Closes-Bug: #1817914

Change-Id: I985d1a7750c18d42042d689c56efee90c20b809c
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2019-02-27 15:39:02 +01:00
Saravanan KR
5c2e741da3 Revert "Remove host-config-and-reboot interface"
This reverts commit 2a8719960103d2619e908290d80235e6079c334d.

BootParams takes care of the kernel args configuration and
reboot via ansible. But Enabling of OVS-DPDK (EnableDpdkDeployment)
is still handled as part of the PreNetworkConfig (host_config_and_reboot).
It can be cleaned-up to remove unused bits, but not the entire stack.

Change-Id: Ifeb029ddaeb2bc60f43abdb8bdb02a4c911ace30
2019-02-20 09:59:08 +05:30
Emilien Macchi
2a87199601 Remove host-config-and-reboot interface
host-config-and-reboot interface was deprecated in Queens in favor of:
OS::TripleO::Services::BootParams: extraconfig/pre_network/boot-params-service.yaml

and doesn't work with config-download, which has been the default since
Rocky.

Let's remove these files.

Change-Id: I85a781cecb7224881b827ff442376c706f1c771b
2019-02-19 11:57:04 -05:00
Jill Rouleau
479d7f5872 flatten qdrouterd service configs
This change combines the previous puppet and docker files
into a single file that performs the docker service installation
and configuration.

Change-Id: I7b1688a12964d939ece33a7f06d2493bf13c08eb
2019-02-15 12:53:15 -07:00
Zuul
d062c5ea56 Merge "Add support for persistence of MariaDB data during reprovisioning" 2019-02-11 19:24:11 +00:00
Jiri Stransky
a64fa251e5 Add support for persistence of MariaDB data during reprovisioning
We should support arbitrary tags in upgrade tasks, update the
validation accordingly.

Change-Id: I3ebeb06b18306a8d1de11b3519e62b90a9cd6a78
Implements: blueprint upgrades-with-os
2019-02-06 14:14:32 +01:00
Cédric Jeanneret
c9adaaeb2f Flatten rabbitmq service - step 2
Now that we could get rid of the puppet dedicated definitions,
we can move the docker/* rabbitmq related stuff to the final
location, and correct the paths and some nits.

Change-Id: I47ca1e303bd38642200ccb7f6823bcd06cd00255
2019-02-06 12:50:51 +01:00
Cédric Jeanneret
fb7ea6734e Flatten rabbitmq service - step 1
This flattens rabbitmq and removes puppet parts. The next step will
move the flattened templates to their final location.

It's split in two steps in order to make reviews easier on that big change.

Change-Id: I30f0802770d86d64e2ec6fa93dc9a608d4b15d69
2019-02-05 15:44:40 +01:00
Zuul
10d612bb12 Merge "UX - Useful error msg if role is not in roles data" 2019-01-31 07:54:14 +00:00
Zuul
70068d220d Merge "Look for parameters in parameter_groups" 2019-01-29 02:35:33 +00:00
Zuul
560ec36685 Merge "Add network data for the undercloud" 2019-01-25 18:05:32 +00:00
Thomas Herve
ae2ccb5f41 Remove workflow_tasks
I don't think it's used anymore.

Change-Id: I928b53d7388e460da3b26306b9f3c548808d329e
2019-01-24 11:08:53 +01:00
Zuul
d385118e8d Merge "Address python3 string issues with subprocess" 2019-01-24 00:36:38 +00:00
Zuul
ac8ebf638c Merge "Remove deprecated TLS-related environment files" 2019-01-23 22:17:10 +00:00
Bogdan Dobrelya
a619d990c0 Address python3 string issues with subprocess
Follows up Id0060a3abbcda8edb6124eb096cb824aaea48396.
This patch updates our Popen calls to enable universal newlines for
calls that we parse or consume the output for. Without
univeral_newlines=True, the output is treated as bytes under python3
which leads to issues later where we are using it as strings.

See https://docs.python.org/3/glossary.html#term-universal-newlines
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>

Change-Id: I1a82c14d8c894cb6ea6c0c322c3fe5b71b34a11e
2019-01-23 16:08:57 +01:00
Harald Jensås
c1116e59c1 Add network data for the undercloud
The undercloud needs the External network for the external VIP
address. It uses the _from_pool template creating a fake neutron
port.

Other networks are not required, and should be removed to optimize
the number of heat resources and plan rendering.

NOTE: Skips validation of network_data_undercloud.yaml against
network_data.yaml, as is already done to openshift and routed
networks networks data examples.

Related Bug: #1809313
Change-Id: Ib11a134df93e59947168b40bc71fb1da9172d4ac
2019-01-21 19:35:37 +01:00
Harald Jensås
53027484ae Skip templating disabled networks
Ignore disabled networks when rendering templates.

Add's the ctlplane network to maps to ensure we don't
end up with no keys/values in map_replace functions.

Also some Jinja cleanup:
 - Reduce the number of times we iterate over networks
   where we can.
 - Add's indentation to make the code easier to read.

Related-Bug: #1809313
Depends-On: I2e8135bc9389d3bf1a6ef01e273515af5c488a9a
Change-Id: Ifeb2d2d1acb43c16a5bf29e95965776494d61fef
2019-01-21 19:35:37 +01:00
Harald Jensås
3fa6349089 UX - Useful error msg if role is not in roles data
If the user specify a role name that does not exist
in the provided roles data the scipt exits with a
StopIteration error. Catch it and raise RuntimeError
with user friendly error message.

Closes-Bug: #1812530
Change-Id: I704316f66c197668a7d8e373efe00889776d2a85
2019-01-20 04:18:13 +01:00
Dan Prince
6fefd102bf Look for parameters in parameter_groups
Look for used parameters in parameter_groups

This patch searches for parameter use in the parameter_groups section
as well and should eliminate some false positive warnings with
our validations.

Change-Id: I67c5ad2b6e865d454489702e5eb263a9508d26d2
2019-01-18 10:09:46 -05:00
Juan Antonio Osorio Robles
a72f8d4ae9 Remove deprecated TLS-related environment files
The ones in environments/ssl/ are preferred instead. These have been
available since pike.

Change-Id: I84a7b354ede46d6ec88964e5dcbd5678d89c8c0f
Depends-On: I5a905ec7499a6faa08cbcacfccb19a6e424e4a80
2019-01-18 09:57:48 +00:00
Dan Prince
144b74d3b4 Look for used parameters in conditionals
This patch searches for parameter use in conditions as well
and should eliminate a lot of false positive warnings with
our validations.

Change-Id: I33eba9d46d8c83b7a34c39fdfcd35b62f52c8752
2019-01-16 13:07:48 -05:00
Harald Jensås
c740b54214 Remove default role-name from merge network param script
The merge-new-params-nic-config-script.py previosly had the
'Controller' role as the default for --role-name. It is not
obvious that this parameter must be changed when merging
nic config templates.

Remove the default and make the argument required. Improves
UX since user error is less likely.

Making the mistake of using a Role with too many networks
is'nt as forgiving since we now only pass parameters for
the role.networks.

Related-Bug: #1800811
Change-Id: Iff9e364db66ad09a30ac10a7814a3c01d50caf58
2019-01-12 13:16:18 +00:00
Zuul
825ae19190 Merge "Designate - Use net_cidr_map for rndc_allowed_addresses" 2019-01-10 21:13:25 +00:00
Zuul
86755894f7 Merge "Apache - Use net_cidr_map for proxy_ips" 2019-01-10 21:13:22 +00:00
Zuul
8f5fb5144d Merge "flatten sshd service configuration" 2019-01-08 06:50:55 +00:00
Zuul
845bc3e845 Merge "Remove MongoDB" 2019-01-07 18:39:49 +00:00
Harald Jensås
5666a4fe0d Designate - Use net_cidr_map for rndc_allowed_addresses
Change I68e064d23ec5d43f59146d974cae604d2c5fdb52 makes
NetCidrMapValue a list of ip networks.

The designate service configures the dns backend security
to contol from wich addresses updates are allowed. We
should use the list of cidr's associated with the
DesignateApiNetwork to allow all nodes in the network to
remotely control the nameserver.

Partial: blueprint tripleo-routed-networks-templates
Change-Id: I5c5cd51c8f127e8879c5528883c3abd261f4a5b3
2019-01-06 18:20:58 +01:00
Harald Jensås
da1de3aafc Apache - Use net_cidr_map for proxy_ips
Change I68e064d23ec5d43f59146d974cae604d2c5fdb52 makes
NetCidrMapValue a list of ip networks.

Pass the list of cidr's from the ApacheNetwork entry in
the cidr map to 'apache::mod::remoteip::proxy_ips:'.

Partial: blueprint tripleo-routed-networks-templates
Change-Id: Ieb6aff9889136f0ccbec32e36b46140aa7826019
2019-01-06 18:20:38 +01:00
Zuul
0e68a0e30f Merge "L3 routed networks - subnet fixed_ips (3/3)" 2019-01-04 20:38:01 +00:00
Emilien Macchi
be07f991b6 Remove MongoDB
MongoDB support was stopped in Pike, it is not used anywhere now.
Therefore, in Stein are removing it to clean things up.

Change-Id: I4ec8f35b1dd71c25cfb41cc54105ac743ef67745
2019-01-04 15:17:00 +00:00
Harald Jensås
2f2d8183e6 L3 routed networks - subnet fixed_ips (3/3)
When using neutron routed networks we need to specify
either the subnet or a ip address in the fixed-ips-request
when creating neutron ports.

a) For the Vip's:

Adds VipSubnetMap and VipSubnetMapDefaults parameters in
service_net_map.yaml. The two maps are merged, so that the
operator can override the subnet where VIP port should be
hosted. For example:

parameter_defaults:
  VipSubnetMap:
    ctlplane: ctlplane-leaf1
    InternalApi: internal_api_leaf1
    Storage: storage_leaf1
    redis: internal_api_leaf1

b) For overcloud node ports:

Enrich 'networks' in roles defenition to include both
network and subnet data. Changes the list to a map
instead of a list of strings. New schema:

- name: <role_name>
  networks:
    <network_name>
      subnet: <subnet_name>

For backward compatibility a conditional is used to check
if the data is a map or not. In either case the internal
list of role networks is created as '_role_networks' in
the jinja2 templates.

When the data is a map, and the map contains the 'subnet'
key the subnet specified in roles_data.yaml is used as
the subnet in the fixed-ips-reqest when ports are created.
If subnet is not set (or role.networks is not a map) the
default will be {{network.name_lower}}_subnet.

Also, since the fixed_ips request passed to Vip ports are no
longer [] by default, the conditinal has been updated to
test for 'ip_address' entries in the request.

Partial: blueprint tripleo-routed-networks-templates
Depends-On: I773a38fd903fe287132151a4d178326a46890969
Change-Id: I77edc82723d00bfece6752b5dd2c79137db93443
2019-01-03 19:07:20 +01:00
Zuul
35f01e48c3 Merge "Process the templates even if j2_excludes file is not present" 2019-01-03 05:43:14 +00:00
David J Peacock
67e74a676c flatten sshd service configuration
This change realigns the sshd baremetal puppet service yaml config
files into a common hierachy as with the rest of this blueprint.

This change also removes container functionality, since this was a
temporary measure to proxy live-migration connections from
non-containerized to containerized compute nodes during upgrade.

Change-Id: I87e112a0f1973fa3b0e959777e00071c2bbf7c9c
Related-Blueprint: services-yaml-flattening
2018-12-19 13:04:08 -05:00
Zuul
1201d46ee3 Merge "YAML NIC Config 2 script - fix comment indentation" 2018-12-12 05:50:32 +00:00
Alex Schultz
623924972f Update yaml-validate for deployment/ folders
Since we're looking at flattening the services into a deployment/
folder, we need to update the validation script to also handle this
directory structure. Additionally this change updates the service name
validation to ensure that the service name in matches the start of the
filename itself.

Change-Id: Ibb140a38b69a8780adf69362e0f437b3426f360d
Related-Blueprint: service-yaml-flattening
2018-12-06 08:06:10 -07:00