Commit Graph

60 Commits

Author SHA1 Message Date
Steven Hardy
87f1f7d26a Add bootstrap_node and vip_data to hierarchy for all roles
We recently made changes that add data to allNodesConfig, but
we didn't wire the files into the hierarchy on all roles.

Change-Id: I8e838b02bd982e600af54b14350106322244890a
Closes-Bug: #1620485
2016-09-07 03:06:25 +00:00
Martin Mágr
25ad7b8e1e Availability monitoring agents support
- adds possibility to install sensu-client on all nodes
- each composable service has it's own subscription

Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Co-Authored-By: Michele Baldessari <michele@redhat.com>
Implements: blueprint tripleo-opstools-availability-monitoring
Change-Id: I6a215763fd0f0015285b3573305d18d0f56c7770
2016-08-31 09:22:59 -04:00
Giulio Fidente
d90363bd58 Move out of roles any Ceph setting
Change-Id: I6ff0f977a1fb2fd55b29becd296ab01c65b50228
2016-08-24 12:51:15 +02:00
Dan Prince
0df577c6f0 Add tripleo-packages composable service
This creates a new service to help manage the puppet-tripleo
class that enables and disables package installation features.

NOTE: we can't move the upgrade setting into the new composable
service yet due to coupling with the UpgradeDeployment resources.

Change-Id: If35cf6a6f023e12ae8ebbc2d9929d244eb3ffa3a
2016-08-17 08:25:37 -04:00
Giulio Fidente
9a6c24beab Remove unneeded net_ip_map dump into hieradata
We will lookup bind addresses using map_replace within the templates
so dumping net_ip_map as hieradata is unneeded.

Change-Id: If54c9033fc58d2cfaa040e30adeed7f58e44fd88
2016-08-16 14:37:06 +00:00
Giulio Fidente
80e82be09b Remove deprecated net_ip_uri_map output
Takes the net_ip_uri_map value from the *_uri values emitted
by net_ip_map instead.

Also removes TenantIp and TenantIpUri from net_vip_map_external
templates as there won't be any VIP on the tenant network.

Change-Id: Icdac3d58162891f5ca3d5c20f14fcdff1781996f
2016-08-16 14:36:47 +00:00
Giulio Fidente
248068bd97 Remove deprecated net_ip_subnet_map output
Change-Id: I83ca923140d7f8ca3101e851e88ca3107a99555a
2016-08-16 14:36:13 +00:00
Steven Hardy
44b67ebdeb Reinstate CloudDomain properties for Controller/Compute roles
These were removed in https://review.openstack.org/#/c/347050
but it turns out the defaults in the role templates is bad, as
an empty string results in a malformed hosts file fqdn.

So, partially revert that patch so we always pass the global
CloudDomain from overcloud.yaml, accepting the default configured
there, and remove the empty-string defaults in the role templates.

Change-Id: I0ea4190a23488986a3ee9e887328e0e7a03fe3aa
2016-08-11 14:04:28 +01:00
Steven Hardy
1d83f18f22 Allow map_replace substitution of network names
To allow per-node data such as bind_ip's to move into the
composable services templates, we do a value substitution
on the config settings hiera map, where e.g internal_api
will be replaced with the NetIpMap IP assigned to that.

To enable subnet/uri lookup via the same method, we add
all the subnet/uri mappings to the main net_ip_map output.

Change-Id: I7850d4dc8bf4db5f7ac6a6b53c1d900b561b4580
2016-08-09 14:32:57 +01:00
Jenkins
173ce97b0c Merge "Remove static hieradata" 2016-08-09 12:39:18 +00:00
Jenkins
6bba45ac28 Merge "Move *Flavor parameters into per-role templates" 2016-08-09 02:45:31 +00:00
Emilien Macchi
cf5fd01b08 Remove static hieradata
Static hieradata moved to composable services, we don't need the files
anymore. It also cleanup how we construct Hieradata configuration by
removing unused hiera files.

Change-Id: I19f85b6c1b734473cf908ddaca29ad966f9f5405
2016-08-08 11:30:12 -04:00
Pradeep Kilambi
7c502ce91d Add Aodh composable roles
Implements: blueprint composable-services-within-roles
Depends-On: Ie48a123cc5bc402aee635a5daf118b158c6f3b6a
Closes-Bug: #1601850

Change-Id: Ifcfe0e3937fa8577635d803d46c3dfc2e873e553
2016-08-05 17:06:08 -04:00
Steven Hardy
1c5a7c3283 Move *Flavor parameters into per-role templates
We have some inconsistent naming here, but move them with their
current names for backwards compatibility, we can address the
deprecation of the inconsistent names at a future time.

This is required to enable jinja templating of roles in overcloud.yaml

Change-Id: I2ea673d9bc52967f9b7c25555059b964abf66966
Partially-Implements: blueprint custom-roles
2016-07-27 17:04:59 +01:00
Steven Hardy
80a1acb55e Move *Image parameters into role templates
We've got some inconsistent naming here, but I'm not attempting to
fix that yet, only move the current parameters inside each role template.

This should be backwards compatible because the parameter names
don't change, but also enable progress on custom-roles.  We can
figure out a strategy for deprecating these and aligning per-role
parameter naming in a subsequent patch.

Also moves ImageUpdatePolicy, which wasn't consistently passed to
all roles anyway, and aligns the default image and constraints
for each role.

Change-Id: I85ec979934df220acbab9f7c3a6055f23e3bfc29
Partially-Implements: blueprint custom-roles
2016-07-27 17:04:59 +01:00
Steven Hardy
466274e287 Move per-role *SchedulerHints parameters into role templates
To enable custom roles, move these into the role templates where
they can be passed via parameter defaults.  Because the Compute
role uses an inconsistent NovaCompute naming, these parameters
cannot be generated in overcloud.yaml, so moving them enables
backwards compatibility to be maintained when we move to a
fully jinja generated overcloud (e.g including the role
ResourceGroup resources)

Change-Id: I3f9b2275f2b1daeb8b83f09548a089dadcfe9eee
Partially-Implements: blueprint custom-roles
2016-07-27 17:04:59 +01:00
Dan Prince
6b30ff11d4 Add 'service_name' to composable services
This patch adds a new service_name section to each composable
service. We now have an explicit unit test check to ensure that
service_name exists in tools/yaml-validate.py.

This patch also wires service_names into hieradata on each
of the roles so that tools can access the deployed services locally
during deployment and upgrades.

Change-Id: I60861c5aa760534db3e314bba16a13b90ea72f0c
2016-07-22 07:29:39 -04:00
Jenkins
a6648b2cfb Merge "Combine BootstrapNodeDeployment with AllNodesDeployment" 2016-07-12 18:36:04 +00:00
James Slagle
c3d595c49a Use already Deployed/Installed servers
This patch provides a set of templates that enables
tripleo-heat-templates to be used with a set of already deployed,
installed, and running servers. In this method, Nova and Ironic are not
used to deploy any servers.

This approach is attractive for POC deployments where dedicated
provisioning networks are not available, or other server install methods
are dictated for various reasons.

There are also assumptions that currently have to be made about the software
installed on the already deployed servers.  Effectively, they must match the
standard TripleO overcloud-full image.

Co-Authored-By: Steve Hardy <shardy@redhat.com>

Change-Id: I4ab1531f69c73457653f1cca3fe30cc32a04c129
2016-07-11 16:20:07 -04:00
Jenkins
7212fd767c Merge "Set os-refresh-config timeout to 4 hours" 2016-07-11 09:30:42 +00:00
Jenkins
869a88d047 Merge "Template param for what command occ runs" 2016-07-11 09:16:37 +00:00
Steven Hardy
b345dbea16 Combine BootstrapNodeDeployment with AllNodesDeployment
Currently we have a special controller-only deployment which writes
the name/ip of the "bootstrap node", e.g the cluster master, which
defaults to the first node in the Controller ResourceGroup.

Now we're moving to fully composable services/roles, it's possible
folks will want to deploy services that expect to detect the bootstrap
node (e.g so only one node does a DB sync) for non-controller roles.

So, take this opportunity to combine the bootstrap node deployment with
the "all nodes" data, such that we deploy the same data for all roles.

Because the boostrap node data is per role cluster, rather than truly
global, we pass it via input_values into each per-role Deployment.

At some future point we might consider renaming this, e.g to
something which describes per-cluster config vs "all nodes",
but as a first step let's just rationalize the resources.

Change-Id: I4011526a13c51b3d0f95c17fe8ed38115b4fdce4
2016-07-05 10:58:31 +01:00
Steve Baker
301560b35a Set os-refresh-config timeout to 4 hours
This change uses the new os-refresh-config --timeout argument to set a
kill timeout for stalled os-refresh-config runs.

4 hours is a reasonable conservative value since it matches the stack
timeout - but it can be set shorter in the future based on actual run
times.

Change-Id: I433f558515df24736263ec0d50de08ad8f78498f
Closes-Bug: #1595722
DependsOn: Ibcbb2090aed126abec8dac49efa53ecbdb2b9b2c
2016-07-05 11:53:23 +12:00
Steve Baker
072b0670cc Template param for what command occ runs
The ConfigCommand parameter overrides the server resource metadata to
specify what command os-collect-config runs whenever any configuration
data changes.

The default is already 'os-refresh-config' so this change has no
effect but it allows a future change to specify an
os-refresh-config --timeout argument to fix bug #1595722.

Change-Id: I8dd35b6724d8c00e5495faca84ee8fee77641b82
Partial-Bug: #1595722
2016-07-05 11:31:39 +12:00
Steven Hardy
59b6afcd35 Replace NodeConfigIdentifiers with DeployIdentifier
We added NodeConfigIdentifiers to trigger config to be re-applied on
update, but then later added DeployIdentifier which forces config to
*always* be applied on update, so we can simplify things by just
referencing the DeployIdentifier directly.

Change-Id: I79212def1936740825b714419dcb4952bc586a39
2016-07-04 21:24:16 +01:00
Giulio Fidente
794fece5cc Switch Ceph Monitor/OSD/Client/External to composable roles
Change-Id: I1921115cb6218c7554348636c404245c79937673
Depends-On: I7ac096feb9f5655003becd79d2eea355a047c90b
Depends-On: I871ef420700e6d0ee5c1e444e019d58b3a9a45a6
2016-07-04 16:38:40 +02:00
Carlos Camacho
c5ea6e699c Composable Timezone service - CephStorage
Add timezone as a composable service

Change-Id: I1569b2ebdca8e67c0e92a5c0e3fadd12006cc02a
Partially-implements: blueprint composable-services-within-roles
2016-06-30 22:54:07 +00:00
Carlos Camacho
94166fb66a Composable roles within services - NTP - CephStorage
Add NTP as a composable service for CephStorage.

Partially-implements: blueprint composable-services-within-roles

Change-Id: Iee89b261b3c45f596ad84549e25d47b8ca1cfbdb
2016-06-21 18:45:51 +02:00
Giulio Fidente
ddd2eb9f13 Configure CephStorage services via resource chains
Also wires in the steps into the CephStorage role.

Change-Id: Ib472f1279478ad7792349cc32bb3c5f510ba69fe
2016-05-26 17:50:49 +02:00
Giulio Fidente
947f47ce90 Dump IPs configuration as hieradata
This might be useful if we switch to %{hiera()} calls to lookup
the bind address from within a service.

Also gets rid of NetIpSubnetMap and provides same output from
NetIpMap instead.

Change-Id: I328a417d1f1fff9c31e9ad7b2b5083ac19bc7329
2016-05-18 19:18:33 +02:00
Giulio Fidente
34b4954bc4 Use 'deeper' hiera_hash merge behavior for all roles
This change configures the hiera merge behavior to 'deeper' [1],
which is useful to merge values when the same hiera key is found
in multiple datafiles.

The hiera default 'native' only picks the value from the key with
the highest priority in the hierarchy.

1. https://docs.puppetlabs.com/hiera/1/lookup_types.html#deep-merging-in-hiera--120

Change-Id: I88c764d9af510ffbbad9fcaa4b747655e38255c2
2016-04-21 12:01:07 +00:00
Juan Antonio Osorio Robles
6d8eb35406 Change /etc/hosts format and add domain
Right now, the service-related IPs assosiated with the machine are
registered in the /etc/hosts with different hostnames. This is fine,
except if you need to register that hostname in a third party service
(such as FreeIPA), since the current configuration is not assigning a
domain to those IP addresses. So the current implementation requires
DNS to be properly working, which is not ideal for testing purposes.

Since the current hostnames are not currently being used; it's still
trivial to change this mapping and the format of them. instead of
having entries such as:

    <INTERNAL IP>  <node>-internalapi
    <STORAGE IP>   <node>-storage
    ...

in /etc/hosts; This changes the format to:

    <INTERNAL IP>  <node>.internalapi.<domain>  <node>.internalapi
    <STORAGE IP>   <node>.storage.<domain>  <node>.storage
    ...

So the network (external, internal, storage, etc...) is now
represented as a subdomain. For simplicity, the format without the
domain is still available through an alias.

Change-Id: I6502959a974546e5de757935acea15df6326acda
2016-04-04 13:56:58 +03:00
Ben Nemec
0b312992e6 Enable predictable IPs on non-controllers
For the external loadbalancer work, we added the ability to specify
fixed ips for controller nodes on all network isolation networks.
In order to allow users full control over the placement and ip
addresses of deployed nodes, we need to be able to do the same thing
for the other node types.

Change-Id: I3ea91768b2ea3a40287f2f3cdb823c23533cf290
2016-03-09 13:25:02 -06:00
Dan Sneddon
321e605a0a Add IPv6 Support to Isolated Networks
This change adds a new set of network templates with IPv6 subnets
that can be used instead of the existing IPv4 networks. Each network
can use either the IPv4 or IPv6 template, and the Neutron subnet will
be created with the specified IP version.

The default addresses used for the IPv6 networks use the fd00::/8
prefix for the internal isolated networks (this range is reserved
for private use similar to 10.0.0.0/8), and 2001:db8:fd00:1000::/64
is used as an example default for the External network
(2001:db8::/32 are the documentation addresses [RFC3849]), but this
would ordinarily be a globally addressable subnet. These
parameters may be overridden in an environment file.

This change will require updates to the OpenStack Puppet
Modules to support IPv6 addresses in some of the hieradata values.
Many of the OPM modules already have IPv6 support to support IPv6
deployments in Packstack, but some OPM packages that apply only to
Instack/TripleO deployments need to be updated.

IPv6 addresses used in URLs need to be surrounded by brackets in
order to differentiate IP address from port number. This change
adds a new output to the network/ports resources for
ip_address_uri, which is an IP address with brackets in the case
of IPv6, and a raw IP address without brackets for IPv4 ports.
This change also updates some URLs which are constructed in Heat.

This has been tested and problems were found with Puppet not
accepting IPv6 addresses. This is addressed in the latest Puppet.
Additional changes were required to make this work with Ceph.

IPv6 tunnel endpoints with Open vSwitch are not yet supported
(although support is coming soon), so this review leaves the
Tenant network as an isolated IPv4 network for the time being.

Change-Id: Ie7a742bdf1db533edda2998a53d28528f80ef8e2
2016-03-04 14:34:59 +01:00
Jenkins
0ed6bb6be8 Merge "Add HostnameMap to allow granular control of hostnames" 2016-03-02 07:15:22 +00:00
Giulio Fidente
d6c1173120 Emits a different hostname for each network the node is on
Populates /etc/hosts with an entry for each IP address the node
is on, which will be useful to migrate services configuration from
using IPs into using hostnames.

This is how the lines look like on a host which doesn't have all ports:

  172.16.2.6 overcloud-novacompute-0.localdomain overcloud-novacompute-0
  192.0.2.9 overcloud-novacompute-0-external
  172.16.2.6 overcloud-novacompute-0-internalapi
  172.16.1.6 overcloud-novacompute-0-storage
  192.0.2.9 overcloud-novacompute-0-storagemgmt
  172.16.0.4 overcloud-novacompute-0-tenant
  192.0.2.9 overcloud-novacompute-0-management

the network against which the default (or primary) name is resolved
can be configured (for computes) via ComputeHostnameResolveNetwork

Change-Id: Id480207c68e5d68967d67e2091cd081c17ab5dd7
2016-02-24 17:56:25 +00:00
Steven Hardy
3c1850b780 Add HostnameMap to allow granular control of hostnames
Some operators desire more granular control of hostnames than is
currently possible via the *HostnameFormat parameters, in particular
mapping nodes to explicit IDs (such as inventory references) is not
easily possible.

So, add a HostnameMap parameter, which is optional and allows
explicit overriding of the default hostnames.

E.g pass an environment like this:

parameter_defaults:
  HostnameMap:
    overcloud-controller-0: overcloud-controller-prod-123-0
    overcloud-controller-1: overcloud-controller-prod-456-0
    overcloud-controller-2: overcloud-controller-prod-789-0

Note this is mapping is global (for all roles), because we
expect the keys to be unique given that they include the
role name and index by default.

Note that this depends on a fix for heat bug #1539737

Change-Id: Ib4d3d40e9523903ebccc06c3e14b2d71d924afa3
Depends-On: Ib934f443a8b8e4f75335a9d8b992e7f86791aa45
2016-02-03 10:17:11 +00:00
Jenkins
795e4290dd Merge "Set the name property for all deployment resources" 2016-01-18 23:16:15 +00:00
Nico Auv
d2c375bad7 Add TimeZone parameter for all node types
Adds a TimeZone parameter for node types and the top level
stack. Defaults to UTC.

Change-Id: I98123d894ce429c34744233fe3e631cbdd7c12b5
Depends-On: Icf7c681f359e3e48b653ea4648db6a73b532d45e
2016-01-05 13:35:48 +01:00
Jenkins
0f42bc2528 Merge "Add all isolated networks to all nodes." 2015-12-23 17:29:37 +00:00
Jenkins
2a05569f02 Merge "Add Management Network For System Administration." 2015-12-23 17:27:25 +00:00
Jenkins
92c994deb8 Merge "Add SoftwareConfigTransport for switching transports" 2015-12-21 20:27:10 +00:00
Dan Sneddon
41b3682f65 Add all isolated networks to all nodes.
This change allows every overcloud node to optionally participate in
any of the isolated networks. The optional networks are not enabled
by default, but allow additional flexibility. Since the new networks
are not enabled by default, the standared deployment is unchanged.
This change was originally requested for OpenDaylight support.

There are several use cases for using non-standard networks.
For instance, one example might be adding the Internal API network
to the Ceph nodes, in order to use that network for administrative
functions. Another example would be adding the Storage Management
network to the compute nodes, in order to use it for backup. Without
this change, any deviation from the standard set of roles that use a
network is a custom change to the Heat templates, which makes
upgrades much more difficult.

Change-Id: Ia386c964aa0ef79e457821d8d96ebb8ac2847231
2015-12-18 13:05:54 -06:00
Dan Sneddon
15bb67261a Add Management Network For System Administration.
This change adds a system management network to all overcloud
nodes. The purpose of this network is for system administration,
for access to infrastructure services like DNS or NTP, or for
monitoring. This allows the management network to be placed on a
bond for redundancy, or for the system management network to be
an out-of-band network with no routing in or out. The management
network might also be configured as a default route instead of the
provisioning 'ctlplane' network.

This change does not enable the management network by default. An
environment file named network-management.yaml may be included to
enable the network and ports for each role. The included NIC config
templates have been updated with a block that may be uncommented
when the management network is enabled.

This change also contains some minor cleanup to the NIC templates,
particularly the multiple nic templates.

Change-Id: I0813a13f60a4f797be04b34258a2cffa9ea7e84f
2015-12-18 13:05:54 -06:00
Jenkins
55e0c425f5 Merge "Fix wrong keypair parameter description" 2015-12-14 22:48:51 +00:00
Steve Baker
572903cd21 Add SoftwareConfigTransport for switching transports
This change adds a SoftwareConfigTransport parameter to role templates
so that the transport can be changed via a parameter_defaults entry.

This change will have no effect on an existing overcloud as the current
default POLL_SERVER_CFN is now explicit in the parameter default.

Change-Id: I5c2a2d2170714093c5757282cba12ac65f8738a4
2015-12-12 12:40:42 +13:00
Steven Hardy
35477d36d3 Fix wrong keypair parameter description
The parameters have nothing to do with EC2 keypairs, they are used to
specify Nova SSH key pairs.

Change-Id: Ia8d37cb5c443812d02133747cb54fcaf0110d091
2015-12-10 10:40:16 +00:00
Steve Baker
1733d74392 Set the name property for all deployment resources
There are two reasons the name property should always be set for deployment
resources:
- The name often shows up in logs, files and API calls, the default
  derived name is long and unhelpful
- Sorting by name determines the merge order of os-apply-config, and the
  execution order of puppet/shell scripts (note this is different to
  resource dependency order) so leaving the default name results in an
  undetermined order which could lead to unpredictable deployment of
  configs

This change simply sets the name to the resource name, but a future change
should prepend each name with a run-parts style 2 digit prefix so that the
order is explicitly stated. Documentation for extraconfig needs to clearly
state what prefix is needed to override which merge/execution order.

For existing overcloud stacks, heat currently replaces deployment resources
when the name changes, so this change
Depends-On: I95037191915ccd32b2efb72203b146897a4edbc9

Change-Id: Ic4bcd56aa65b981275c3d4214588bfc4de63b3b0
2015-12-10 14:48:04 +13:00
Steven Hardy
5d61012f86 Enable per-role SchedulerHints
This adds a parameter for each role, where optional scheduler hints
may be passed to nova.  One potential use-case for this is using
the ComputeCapabilities to pin deployment to a specific node (not
just a specific role/profile mapping to a pool of nodes like we
have currently documented in the ahc-match docs).

This could work as follows:

1. Tag a specific node as "node:controller-0" in Ironic:

ironic node-update <id> replace properties/capabilities='node:controller-0,boot_option:local'

2. Create a heat environment file which uses %index%

parameters:
  ControllerSchedulerHints:
    'capabilities:node': 'controller-%index%'

Change-Id: I79251dde719b4bb5c3b0cce90d0c9d1581ae66f2
2015-12-08 10:28:08 +00:00
Juan Antonio Osorio Robles
a7925bc9ed Add option to add metadata for the overcloud nodes
Some Nova hooks might require custom properties/metadata set for the
servers deployed in the overcloud, and this would enable us to inject
such information.

For FreeIPA (IdM) integration, there is effectively a Nova hook that
requires such data.

Currently this inserts metadata for all servers, but a subsequent CR
will introduce per-role metadata. However, that was not added to this
because it will require the usage of map_merge. which will block those
changes to be backported. However, this one is not a problem in that
sense.

Change-Id: I98b15406525eda8dff704360d443590260430ff0
2015-12-04 15:36:30 +02:00