368 Commits

Author SHA1 Message Date
Jenkins
ab682ed638 Merge "Rename service_workflow_tasks into workflow_tasks" 2017-09-14 17:01:44 +00:00
Marius Cornea
e471c67aab Remove deploy_steps_tasks.yaml from upgrade_steps_playbook
After landing https://review.openstack.org/#/c/503484/ we run the
puppet host configuration steps twice. This change removes the
deploy_steps_tasks.yaml playbook in order to run the puppet steps
only once.

Closes-bug: 1717244
Change-Id: I09461094618124915841c8390c8bce8daf64d029
2017-09-14 14:01:52 +02:00
Jenkins
3c6f26e890 Merge "Revert "Tag workflows created by the templates"" 2017-09-13 17:46:39 +00:00
Giulio Fidente
09137304b9 Rename service_workflow_tasks into workflow_tasks
Using the service_ prefix seems incoherent with its use in
service_config_settings (vs config_settings).

Change-Id: Ia39f181415bee0071409dabddfa0c5c312915e1f
2017-09-13 17:15:17 +02:00
Alex Schultz
ab7fd80008 Revert "Tag workflows created by the templates"
This reverts commit a7a02f0da866c66dce9757a42bf56144cfa70d5a.

This change requires a heat functionality which is not yet available so scenario001-containers job fails because of the new tags.  Reverting to unblock CI and this should be back after we have heat promotion

Change-Id: Ib0fed291c1c4e41d1ea0bb7fc2ccbdabac1d336b
Closes-Bug: #1716915
2017-09-13 12:42:28 +00:00
Jenkins
3dcc9b30e9 Merge "Tag workflows created by the templates" 2017-09-13 10:56:28 +00:00
Steven Hardy
27018b4182 Add RoleConfig output to major_upgrade_steps.j2.yaml
I96ec09bc788836584c4b39dcce5bf9b80e914c71 added this output to the
deploy-steps.j2, but missed adding this to the major upgrade template
which means the overcloud RoleConfig output is broken after the upgrade
(until the converge update switches back to the deploy-steps.j2 derived
template)

Closes-Bug: #1716404
Change-Id: I331fa18b456ca2d6c124316d513374e3fe5a5007
2017-09-12 09:58:50 +01:00
Giulio Fidente
a7a02f0da8 Tag workflows created by the templates
This is useful to easily filter workflows created by the templates
and for a specific stack.

Change-Id: I0a26cacaf5ad5709881043434694c9254a9e710b
Related-Bug: #1715389
2017-09-11 17:01:52 +00:00
Steven Hardy
94c7752cfa Set mode for ansible written files
Use a more restrictive mode for these files, as some may contain sensitive data
which shouldn't be world readable

Closes-Bug: #1714986
Change-Id: Ib1e79b1d4e25d6e329938402b1ca776bdab81bdd
2017-09-04 16:38:26 +01:00
Thomas Herve
8008089de2 Use list_concat in place of yaql
Where applicable, use list_concat instead of yaql to build new lists: it
should be more resilient to errors, easier to debug, and less expensive.

Change-Id: I6d3dbc7ee8eac50f46023a35af4ec7f2d378fd87
Related-Bug: #1714005
2017-08-30 15:43:16 +02:00
Dan Prince
949d367dde Add DockerPuppetProcessCount defaults to 3
docker-puppet.py is very aggressive about running concurrently.
It uses python multiprocessing to run multiple config generating
containers at once. This seems to work well in general, but
in some cases... perhaps when the registry is slow or under
heavy load can cause timeouts to occur. Lately I'm seeing
several 'container did not start before the specified timeout'
errors that always seem to occur when config files are generated
(docker-puppet.py is initially executed.

A couple of things:

 -when config files are generated this is the first time
  most of the containers are pulled to each host machine
  during deployment

 -docker-puppet.py runs many of these processes at once. Some
  of them run faster, other not.

 -docker daemon's pull limit defaults to 3. This would throttle
  the above a bit perhaps contributing the the likelyhood of a timeout.

One solution that seems to work for me is to set the PROCESS_COUNT
in docker-puppet.py to 3. As this matches docker daemon's default
it is probably safer at the cost of being slightly slower in some
cases.

Change-Id: I17feb3abd9d36fe7c95865a064502ce9902a074e
Closes-bug: #1713188
2017-08-25 23:01:24 -04:00
Mathieu Bultel
d9d8314d26 Specify the start count to 0 for the update step loop
Force the count start to 0 to ensure that the
update step loop will start to 0 and execute the
update step0

Closes-Bug: #1712498

Change-Id: I71be55c1f56e53e5c565bec281795d63e5845ff6
2017-08-23 08:25:57 +00:00
marios
060ff37c4f Also write an upgrade_tasks_playbook
To get this to work upgrade_tasks need to be rewritten with 'when'
statements like the update tasks (in parent review from shardy).
So that we don't break the existing upgrades workflow, we add these
as part of the config download see the depends on

Related-Bug: 1708115
Depends-On: Ief593dc758a2ffe33c1cbcbda9289393fcf023e4
Change-Id: Ib01b96a2c26721747d81d98e3d57c4c388663004
2017-08-15 11:48:48 +00:00
Steven Hardy
ec58a4b6c5 Add environment to disable deploy steps
This enables either deploying without configuring any services, or
temporarily disabling the deploy steps such as will be required
for minor updates where we want to re-run the rolling update outside
of heat.

To deploy directly via ansible-playbook you can do e.g:

openstack overcloud config download --config-dir tmpconfig
cd tmpconfig/tripleo-6b02U7-config
ansible-playbook -vvv -b -i /usr/bin/tripleo-ansible-inventory deploy_steps_playbook.yaml

Which will run the same ansible steps as we normally run via heat.

Change-Id: I59947b67523dfcc43d454d4ac7d82b06804cf71d
2017-08-12 10:40:57 +00:00
Steven Hardy
1801565a75 Add support for update_tasks
These work the same way as upgrade_tasks *but* they use a step variable
instead of tags, so we can iterate over a count/sequence which isn't
possibly via a wrapper playbook with tags (we may want to align upgrade
tasks with the same approach if this works out well).

Note the tasks can be run via ansible-playbook on the undercloud, like:

openstack overcloud config download --config-dir tmpconfig
cd tmpconfig/tripleo-HCrDA6-config
ansible-playbook -b -i /usr/bin/tripleo-ansible-inventory update_steps_playbook.yaml --limit controller

The above will do a rolling update for the Controller role (note the inconsistent
capitalization, we probably need to fix the group naming in tripleo-ansible-inventory)
because we specify serial: 1 in the playbook.

You can also trigger an update explicitly on one node like this, which is useful for debugging:

ansible-playbook -vvv -b -i /usr/bin/tripleo-ansible-inventory update_steps_playbook.yaml --limit overcloud-controller-0

Change-Id: I20bb3e26ab9d9cadf1a31fd304de8a014a901aa9
2017-08-12 10:40:48 +00:00
Steven Hardy
46279be9cb Add RoleConfig output
This exposes the deploy workflow for all roles from deploy-steps
via overcloud.j2.yaml - which means we can write it via the new
openstack overcloud config download command and/or run the workflow
outside of heat via mistral

With https://review.openstack.org/#/c/485732/ applied to
tripleoclient it becomes possible to do:

openstack overcloud config download --config-dir tmpconfig
cd tmpconfig/tripleo-EvEZk0-config
ansible-playbook -b -i /usr/bin/tripleo-ansible-inventory deploy_steps_playbook.yaml

This runs the deploy steps, exactly the same as normally run via heat
via ansible-playbook for all overcloud nodes (--limit can be used to restrict
to specific nodes/roles).

Change-Id: I96ec09bc788836584c4b39dcce5bf9b80e914c71
2017-08-12 10:40:41 +00:00
Steven Hardy
76421eb249 Move deploy-steps-playbook to deploy-steps-tasks
So that we can more easily iterate over an include in an output

Change-Id: Idd5bb47589e5c37123caafcded1afbff8881aa33
2017-08-12 10:40:25 +00:00
Steven Hardy
7f6305980d Consolidate puppet/docker deployments with one deploy steps workflow
If we consolidate these we can focus on one implementation (the new ansible
based one used for docker-steps)

Change-Id: Iec0ad2278d62040bf03613fc9556b199c6a80546
Depends-On: Ifa2afa915e0fee368fb2506c02de75bf5efe82d5
2017-08-11 17:25:02 +00:00