68 Commits

Author SHA1 Message Date
Zuul
950640ad52 Merge "Use templating for nova cell database_connection" 2019-01-07 14:02:29 +00:00
Harald Jensås
2f2d8183e6 L3 routed networks - subnet fixed_ips (3/3)
When using neutron routed networks we need to specify
either the subnet or a ip address in the fixed-ips-request
when creating neutron ports.

a) For the Vip's:

Adds VipSubnetMap and VipSubnetMapDefaults parameters in
service_net_map.yaml. The two maps are merged, so that the
operator can override the subnet where VIP port should be
hosted. For example:

parameter_defaults:
  VipSubnetMap:
    ctlplane: ctlplane-leaf1
    InternalApi: internal_api_leaf1
    Storage: storage_leaf1
    redis: internal_api_leaf1

b) For overcloud node ports:

Enrich 'networks' in roles defenition to include both
network and subnet data. Changes the list to a map
instead of a list of strings. New schema:

- name: <role_name>
  networks:
    <network_name>
      subnet: <subnet_name>

For backward compatibility a conditional is used to check
if the data is a map or not. In either case the internal
list of role networks is created as '_role_networks' in
the jinja2 templates.

When the data is a map, and the map contains the 'subnet'
key the subnet specified in roles_data.yaml is used as
the subnet in the fixed-ips-reqest when ports are created.
If subnet is not set (or role.networks is not a map) the
default will be {{network.name_lower}}_subnet.

Also, since the fixed_ips request passed to Vip ports are no
longer [] by default, the conditinal has been updated to
test for 'ip_address' entries in the request.

Partial: blueprint tripleo-routed-networks-templates
Depends-On: I773a38fd903fe287132151a4d178326a46890969
Change-Id: I77edc82723d00bfece6752b5dd2c79137db93443
2019-01-03 19:07:20 +01:00
Oliver Walsh
7288062676 Use templating for nova cell database_connection
Nova now allows use of templated urls in the database and mq
connections which will allow static configuration elements to be
applied to the urls read from the database per-node. This should
be a simpler and less obscure method of configuring things like
the per-node bind_address necessary for director's HA arrangement.

This patch addresses the templated DB urls as part 1.

Nova support added here:
https://review.openstack.org/#/c/578163/

Related-Bug: 1808134

Co-Authored-By: Martin Schuppert <mschuppert@redhat.com>

Change-Id: If30b4647bca210663a22fd653e752d4d57345bdd
2018-12-20 16:30:00 +01:00
Zuul
769f18f0f5 Merge "Check for available networks for a role" 2018-12-05 19:01:14 +00:00
Rabi Mishra
5d275fb922 Check for available networks for a role
For network isolation, we specifcy available networks for role.
Therefore, there is no point in creating noop network resources for
networks that are not available/connected. This results in redundant
host entries for not available networks on overcloud nodes.

If a network is not available for a role we don't need to create
those extra noop resources.

For Undercloud/Standalone role we keep all networks in roles data
as the default ServiceNetMap specifies non ctlplane networks though
they map to ctlplane.

Change-Id: I07822ec0cba7eed352c0010eb893b5e5a522e95c
Closes-Bug: #1800811
2018-11-19 10:14:34 +05:30
Alex Schultz
653649ebbc Add OpenStack clients service
We did not have a easy way to ensure all the openstack clients are
installed on a given system. In the old instack-undercloud installation,
we were installing some additional clients outside of the ones required
via python-tripleoclient. To allow a user to quickly install all the
clients on a given system, this change adds an OpenStack clients
"service" which can be added to a role to ensure the clients are
available. In the future if we provide a client container, this service
can be converted into a container deployment mechanism.

Change-Id: If878c2ab7679eea2fff42b410bec9c8c9b92ed6f
Closes-Bug: #1800001
2018-10-26 16:25:35 -06:00
Michele Baldessari
c2139a7db2 Fix TLS when using a containerized undercloud
Since we moved to containerized UC, TLS Everywhere deployments are broken.
Namely we miss two things:

A. The NAT iptables rule for the nova metadata service to be reachable
B. The setting 'service_metadata_proxy=false' needs to be set for nova
   metadata otherwise the curl calls to setup ipa will fail with the
   following:
[root@overcloud-controller-0 log]# curl http://169.254.169.254/openstack/2016-10-06
<html>
 <head>
  <title>400 Bad Request</title>
 </head>
 <body>
  <h1>400 Bad Request</h1>
  X-Instance-ID header is missing from request.<br /><br />
 </body>
</html>

A. Is fixed by adding a conditional iptables rule that is only triggered
   when deploying an undercloud (where we set MetadataNATRule to true)

B. Is fixed by setting NeutronMetadataProxySharedSecret to '' on the
   undercloud and then setting the corresponding hiera keys only when
   the parameter != ''. We tried alternative simpler approaches like
   setting NeutronMetadataProxySharedSecret to null but that will break
   heat as the parameter is required and setting it to null breaks heat
   validation (we also tried to make the parameter optional with a
   default: '', but that broke as well)

While we're at it we also remove the neutron metadata service from the
undercloud as it is not needed.

Tested by deploying an undercloud with this change and observing:
A.
Chain PREROUTING (policy ACCEPT 106 packets, 6698 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 REDIRECT   tcp  --  br-ctlplane *       0.0.0.0/0            169.254.169.254      multiport dports 80 state NEW /* 999 undercloud nat ipv4 */ redir ports 8775

B.
grep -ir ^service_metadata_proxy /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf
service_metadata_proxy=False

Also a deployment of a TLS overcloud was successful.

Change-Id: Id48df6db012fb433f9a0e618d0269196f4cfc2c6
Co-Authored-By: Martin Schuppert <mschuppe@redhat.com>
Closes-Bug: #1795722
2018-10-06 13:25:18 +00:00
Emilien Macchi
7bebdefda8 Introduce OS::TripleO::Services::Podman
Podman service will be in charge of installing, configuring, upgrading
and updating podman in TripleO.

For now, the service is disabled by default but included in all roles.
In the cycle, we'll make it the default.

Note: when Podman will be able to run in TripleO without Docker,
we'll do like https://review.openstack.org/#/c/586679/ and make it as
a generic service that can be switched to either podman or docker.
But for now, we need podman & docker working side by side.

Depends-On: Ie9f5d3b6380caa6824ca940ca48ed0fcf6308608
Change-Id: If9e311df2fc7b808982ee54224cc0ea27e21c830
2018-10-02 01:47:46 +00:00
Zuul
58d3eec514 Merge "Create a Timesync service declaration" 2018-09-18 21:29:21 +00:00
Alex Schultz
f7f9053963 Create a Timesync service declaration
In order to support switching between multiple timesync backends, let's
simplify the service configurations for the roles so that there is a
single timesync service.  This timesync service should point to the
expected backend (ntp/ptp/chrony).

Change-Id: I986d39398b6143f6c11be29200a4ce364575e402
Related-Blueprint: tripleo-chrony
2018-09-04 21:00:56 +00:00
Emilien Macchi
d2f410ea73 undercloud: deploy kernel composable service
To benefic from syctl parameters interface, we need this service on the
containerized undercloud.

Change-Id: I59d6198fa53ee3b3bf375bc7bfb7caa880146c62
2018-09-04 20:17:38 +00:00
Zuul
39e360624e Merge "Break out image prepare into its own "service"" 2018-08-10 02:31:33 +00:00
Juan Antonio Osorio Robles
60a63edc22 Add neutron metadata agent to the undercloud
It was missing and it's necessary for the nova metadata API to work.
Without this we don't have a working TLS everywhere setup, since it
relies on this functionality.

Change-Id: I24ff6f1b5acc428f001b2ca9b0bdbfa8ec121e52
Closes-Bug: #1785744
2018-08-07 08:17:08 +03:00
Steve Baker
1bda1fd9a7 Break out image prepare into its own "service"
This makes the docker-registry service focused on installing the
registry, as it should be. Also this makes it possible to invoke this
service during overcloud deploy too.

This change also switches to calling the tripleo-common script
tripleo-container-image-prepare instead of the full openstack command.
This will allow a mistral image to do a prepare without depending on
the python-tripleoclient package.

The {{role}}Services and {{role}}Count are propagated to
tripleo-container-image-prepare so that images are filtered correctly.

sudo is used instead of become:true so that the tripleo-common mistral
sudoers pattern matches.

Depends-On: Ic1648e43f45bb7604d4c0f9abf247a475fb23707
Change-Id: Ibc16bed673de7b22cd8eef3f6fb0d45871083873
Blueprint: container-prepare-workflow
2018-08-02 11:29:39 +12:00
Cédric Jeanneret
6237903852 Allow to remove xinetd service
The xinetd service isn't used anymore on the host - it runs in
containers where it is needed, meaning that service can be dropped
for good, as well as its package.

Change-Id: I004a43c1b6c9cee21c24749bd6589435530e48e0
2018-07-31 15:21:59 +02:00
Alex Schultz
db181732c6 Add SELinux management to containerized undercloud
In instack-undercloud we manage the selinux configuration during the
deployment. This change exposes the configuration as a new tripleo
service for selinux so we can configure it.

Change-Id: I2109bf62e307df92b6bdb57600c58dd61482f46d
Partial-Bug: #1779005
2018-06-28 09:12:30 -06:00
Emilien Macchi
0e0147732c Implement TripleoValidations composable service
Implement a service that will deploy TripleO Validations package and
user by using Puppet like we did with instack-undercloud. This service
will be included on the undercloud but disabled by default. We'll keep
the same interface in undercloud.conf to enable it or not so no change
for the end user.

Change-Id: Ida09f92010e31d952edd82b42a7fc20451537d42
2018-06-13 11:35:49 -07:00
Alex Schultz
64bc4a7683 Remove undercloud specific service definitions
Since we're aligning the overcloud/undercloud and we've switched to
containerization it, we should reuse the same heat services rather than
duplicating the services with the Undercloud definition.

Depends-On: Ic7dba7e548f85574cce2db23e3fec5c8ea761bb7
Change-Id: I497597a47533375f34a22a56e2e9a145d9393358
Related-Blueprint: containerized-undercloud
2018-05-09 21:33:20 +00:00
Zuul
822bd996b3 Merge "Support separate oslo.messaging services for RPC and Notification" 2018-04-25 04:43:46 +00:00
Andrew Smith
78bc457585 Support separate oslo.messaging services for RPC and Notification
This commit introduces oslo.messaging services in place of a single
rabbitmq server. This will enable the separation of rpc and
notifications for the continued use of a single backend (e.g.
rabbitmq server) or a dual backend for the messaging communications.

This patch:
* add oslo_messaging_rpc and oslo_messaging_notify services
* add puppet services for rpc and notification
  (rabbitmq and qdrouterd servers)
* add docker services to deploy rpc (rabbitmq or qdrouterd)
  and notify (rabbitmq or shared)
* retains rabbit parameters for core services
* update resource registries, service_net_map, roles, etc.
* update ci environment container scenarios
* add environment generator for messaging
* add release note

Depends-On: Ic2c1a58526febefc1703da5fec12ff68dcc0efa0
Depends-On: I154e2fe6f66b296b9b643627d57696e5178e1815
Depends-On: I03e99d35ed043cf11bea9b7462058bd80f4d99da
Needed-By: Ie181a92731e254b7f613ad25fee6cc37e985c315
Change-Id: I934561612d26befd88a9053262836b47bdf4efb0
2018-04-22 04:33:44 +00:00
Zuul
0bf8943f05 Merge "undercloud: deploy SSH service" 2018-04-20 11:11:47 +00:00
Zuul
4e1e6528c7 Merge "Add Barbican to the list of services for the undercloud" 2018-04-19 10:06:43 +00:00
Emilien Macchi
ae61a031a6 undercloud: deploy SSH service
For parity with non containerized undercloud, add SSH service on the
undercloud.

Change-Id: Ia642c7d54acbc89630f79a8e64e7798c1a25d23f
2018-04-18 19:18:32 -07:00
Zuul
628cd0e390 Merge "Add Ironic Networking Baremetal Templates" 2018-04-18 05:52:20 +00:00
Juan Antonio Osorio Robles
252f56175c Add Barbican to the list of services for the undercloud
It's disabled by default, and will be an option to use if we decide to
enable Swift volume encryption for the undercloud.

Change-Id: I9c5e07a2eb764168670d5de7bdeb4b6362f9bfb5
2018-04-16 15:59:11 +00:00
Emilien Macchi
d86025593b Handle undercloud upgrades via host_prep_tasks
Using host_prep_tasks interface to handle undercloud teardown before we
run the undercloud install.
The reason of not using upgrade_tasks is because the existing tasks were
created for the overcloud upgrade first and there are too much logic
right now so we can easily re-use the bits for the undercloud. In the
future, we'll probably use upgrade_tasks for both the undercloud and
overcloud but right now this is not possible and a simple way to move
forward was to implement these tasks that work fine for the undercloud
containerization case.

Workflow will be:
- Services will be stopped and disabled (except mariadb)
- Neutron DB will be renamed, then mariadb stopped & disabled
- Remove cron jobs
- All packages will be upgraded with yum update.

Change-Id: I36be7f398dcd91e332687c6222b3ccbb9cd74ad2
2018-04-12 18:14:28 -07:00
Harald Jensas
5203e43979 Add Ironic Networking Baremetal Templates
Ironic neutron agent will be installed on controller nodes, or
networker nodes, when environments/services/ironic.yaml or
environments/services-docker/ironic.yaml is used.

It should also be enabled on undercloud.

Also enables ``baremetal`` ML2 mechanism driver on undercloud.

Depends-On: Ic1f44414e187393d35e1382a42d384760d5757ef
Depends-On: I3c40f84052a41ed440758b971975c5c81ace4225
Change-Id: I0b4ef83a5383ff9726f6d69e0394fc544c381a7e
2018-04-12 23:59:34 +02:00
Emilien Macchi
04b898d0c1 Implement MasqueradeNetworks services
To port what has been done in instack-undercloud, we need a new service
to manage IPtables rules when we need to redirect network through the
undercloud via masquerading.
It was done in instack-undercloud via bash, it'll now be done in THT via
a service, disabled by default and that will be activated in CI when
deploying with a containerized undercloud.

Co-Authored-By: Thomas Herve <therve@redhat.com>
Depends-On: Ic9a2626e73d132c3be7ff14a1f4cdba0c16c5b53
Change-Id: I93ff9a3bebcec1bc7ee188f9ec00feafca2c5117
2018-03-27 17:22:02 +02:00
Zuul
7c466fb751 Merge "docker: add support for TripleO UI" 2018-03-17 11:38:01 +00:00
Honza Pokorny
7cbe28b5ef docker: add support for TripleO UI
Co-Authored-By: Martin André <m.andre@redhat.com>
Co-Authored-By: Dan Prince <dprince@redhat.com>
Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Partially-Implements: bp tripleo-ui-undercloud-container
Change-Id: I1109d19e586958ac4225107108ff90187da30edd
2018-03-15 23:43:17 +01:00
Juan Antonio Osorio Robles
781e1b2b4c Add novajoin service
This adds the relevant templates to enable novajoin in a containerized
undercloud environment. Note that this is not meant for the overcloud
(yet), and since there are several limitations that need to be addressed
first. This is meant for the containerized undercloud.

Depends-On: Iea461f66b8f4e3b01a0498e566a2c3684144df80
Depends-On: Ia733b436d5ebd0710253c070ec47a655036e0751
Depends-On: I554125fd6b48e620370f9e3a6061bbdc1d55b0ae
Change-Id: I3aad8a90816e6fc443f20579f6ac7ad4f35eafcb
2018-03-14 13:55:16 +00:00
Chandan Kumar
38387ada75 Added t-h-t for tempest container
It adds a fake tempest service in order to make tempest
container available on undercloud.

Change-Id: If580292572ee5a11a41f68752a6ffe8a99f066e7
2018-03-12 06:42:15 +00:00
Emilien Macchi
05a0f6cdec Add TripleOFirewall service to undercloud roles
... so we can configure IPtables on containerized undercloud.

Depends-On: https://review.openstack.org/545367
Change-Id: I9f8c3d18938926257456388fd15e8eeb2e2868fd
2018-02-17 01:42:06 +00:00
Emilien Macchi
9d9289cf6d undercloud: remove duplicate Neutron Server entry
... or the deployment fails since we try to deploy twice the
OS::TripleO::Services::NeutronApi service.

Change-Id: I92d5d037074494c40fb2b1968985a95ffd2fae12
2018-02-17 01:42:01 +00:00
Pradeep Kilambi
7a5d5a8e1b Add tls roles for undercloud
Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Co-Authored-By: Dan Prince <dprince@redhat.com>
Co-Authored-By: Ian Main <imain@redhat.com>

Change-Id: Icca382db28e4ea57f3cbf24e9e794b428b824db5
2018-02-15 00:00:05 +00:00
lhinds
7e68dbdf8c Implements AIDE Intrusion Detection System
Introduces a service to configure AIDE Intrusion Detection.

This service init's the database and copies the new database
to the active naming. It also sets a cron job, using email if
`AideEmail` is populated, otherwise the reports are sent to
`/var/log/aide/`.

AIDE rules can be supplied as a hash, and should the rules ever
be changed, the service will populate the new rules and re-init
a fresh integrity database.

Related-Blueprint: tripleo-aide-database
Depends-On: Iac2ceb7fc6b610f8920ae6f75faa2885f3edf6eb
Change-Id: I23d8ba2c43e907372fe079026df1fca5fa1c9881
2018-01-15 13:10:16 +00:00
Zuul
ef6d97c543 Merge "Telemetry Needs Redis" 2018-01-08 18:21:24 +00:00
Emilien Macchi
6a6872f390 Introduce OS::TripleO::Services::Rhsm
Background:
extraconfig/pre_deploy/rhel-registration interface has been maintained
for some time now but it's missing some features and the code overlaps
with ongoing efforts to convert everything to Ansible.

Plan:
Consume ansible-role-redhat-subscription from TripleO, so all the logics
goes into the Ansible role, and not in TripleO anymore.
The single parameter exposed to TripleO is RhsmVars and any Ansible
parameter can be given to make the role working.
The parameter can be overriden per roles, so we can think at specific
cases were some Director roles would have specific RHSM configs.
Once we have feature parity between what is done and what was here
before, we'll deprecate the old interface.

Testing:
Because RHSM can't be tested on CentOS, this code was manually tested on
RHEL against the public subscription portal. Also, we verified that
generated Ansible playbooks were correct and called the role with the
right parameters.

Documentation:
We'll work on documentation during the following weeks and explain
how to switch from the previous interface to the new one, and also
document new uses requested by our users.

Change-Id: I8610e4f1f8478f2dcbe3afc319981df914ce1780
2017-12-27 11:03:49 -08:00
Ian Main
b54135fc3a Telemetry Needs Redis
Add redis to the undercloud when telemetry is added.

Change-Id: I5fc235e6f77efba73ab1858e959357a954c7b7a3
2017-12-27 17:29:18 +00:00
Dan Prince
cec41586f7 Add docker-registry service
This is required for the containerized undercloud.

Change-Id: I542a19c084f37aaafd72378857af4f379f335a39
2017-12-27 01:41:50 +00:00
lhinds
502fde7a64 Implements management of /etc/login.defs
Enables management of shadow password directives in login.defs

By allowing operators to set values in login.defs, they are able
to improve password security for newly created system accounts.

This change will in turn allow operators to adhere with security
hardening frameworks, such as STIG DISA & CIS Security Benchmarks.

bp login-defs

Change-Id: Id4fe88cb9569f18f27f94c35b5c27a85fe7947ae
Depends-On: Iec8c032adb44593da3770d3c6bb5a4655e463637
2017-11-29 09:23:25 +00:00
Zuul
4fa81458d4 Merge "Undercloud: Add router for IPv6 ctlplane subnet" 2017-11-27 23:08:23 +00:00
Zuul
f0be3d1f0d Merge "Add Docker service to the undercloud roles" 2017-11-16 15:25:09 +00:00
Dan Prince
ef78b46338 Undercloud: Add router for IPv6 ctlplane subnet
A router is needed so that router advertisements are send out for
the subnet. Also add the router plugin and start the l3 agent
which are responsible for the router handling.

Change-Id: I2c7c6232d00a11f550ad186f94ce628090ec93ed
2017-11-13 14:20:32 -05:00
Pradeep Kilambi
07ce5c4bae Add undercloud cinder roles and environment files
If enable_cinder is true in undercloud.conf, we will need to include
these env files to setup cinder containers.

Change-Id: I208347c52ac5ad24a54aade0be23a31f5bdd4249
2017-11-10 16:39:04 +00:00
Dan Prince
f0b4b0dffe Add Docker service to the undercloud roles
This patch adds the Docker service to the default undercloud
roles data. By default Docker is set to OS::Heat::None.
When using environments/docker.yaml however it will get set
to puppet/service/docker.yaml which will facilitate installing
the Docker service early in the deployment at step 1.

Change-Id: I2d569eef136254dc81bdee93a7869fd361a8400d
2017-11-09 12:18:13 +11:00
Brad P. Crochet
95a7a27fcb Add Mistral event engine
Mistral has an event engine to trigger cron-type events. Let's use it.

Change-Id: I15b48bd7a501608b1fad64fea8d4f9822946dcb2
Depends-On: I71f556c96ed7c2bbafacab4b2f66874effbd8b73
2017-09-28 06:55:46 -04:00
Bogdan Dobrelya
8a03456056 Add logrotate with crond service
Add a docker service template to provide containerized services
logs rotation with a crond job.
Add OS::TripleO::Services::LogrotateCrond to CI multinode-containers
and to all environments among with generic services like Ntp or Kernel.
Set it to OS::Heat::None for non containerized environments and
only enable it to the environments/docker.yaml.

Closes-bug: #1700912

Change-Id: Ic94373f0a0758e9959e1f896481780674437147d
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2017-08-21 08:56:29 +02:00
Joe Talerico
c2b2cc555a Adding Tuned Service
Allow the user to set a specific Tuned profile on a given host.

Defaults to throughput-performance

Change-Id: I0c66193d2733b7a82ad44b1cd0d2187dd732065a
2017-07-25 17:08:37 +00:00
Dan Prince
5b77325687 Drop MongoDB from the undercloud
This patch drops MongoDB from the undercloud and swaps
in a 'swift' and 'sqlalchemy' versions for the Zaqar
backends.

Change-Id: Ic6600f284dfbf4360b42598d3651b4c0f8046d1c
2017-07-23 22:29:02 +00:00