18 Commits

Author SHA1 Message Date
Kevin Carter
e971e15997
Integrated updates after the multi-distro changes
* Corrected the repo server log directory. This change updates the repos
  server log directory which was being incorrectly indexed. The old container
  build process would create a service log directory based on "properties".
  The "properties" log directory is unused or is re-defined by the various
  service roles. This change simply corrects the log paths which the
  `rsyslog_client` role uses to ensure all logs are shipped to the right
  places.
* Add a log directory creation task to the utility container. This play
  incorrectly assumed that the log directory based on "properties" would
  be automatically created. This update simply makes that more explicit.
* paramiko has been pinned to match global requirements and use a version
  <2. This will ressolve runtime issues in Newton with an unbound
  requirements.

Closes-Bug: #1576755
Change-Id: I15d841106ec9a13555b9737c9388f40557f5bec5
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-05-03 22:51:43 -05:00
git-harry
ef347ab4b8 Remove unused var pip_no_index
The var pip_no_index is not used by repo-server.yml or any of its roles
and appears to have been missed by some previous change to the playbook
or the role repo_server. This commit removes this unused var.

Change-Id: Ifdb21cfb05ba412e6959cd53d7bdeeaa2e43fae7
2016-04-28 09:00:19 +01:00
Jimmy McCrory
bcbc5b27fc Remove unneeded playbook vars
ansible_hostname is not used within any tasks and, by default, is the
same value as container_name.

ansible_ssh_host and container_address are also the same value by
default, both are assigned to hosts through the dynamic inventory script.
Additionally, overriding ansible_ssh_host through playbook vars breaks
tasks that delegate to other hosts when using Ansible2.

Change-Id: I2525b476c9302ef37f29e2beb132856232d129e2
2016-01-25 16:21:22 -08:00
Kevin Carter
975c232084 Updated the repo-build process
This commit changes the repo build play such that it no longer
requires "yaprt". The intention here is to make the build process
less of a blackbox, simpler, and faster. With recent improvements
in OpenStack and PIP we're now able to do a lot more without the
use of the "yaprt" tool. Repo building is now its own standalone
role.

This PR now reintroduces the repo-clone ability which is now made
smarter. The repo-build process creates a manifest file which is
used to provide the ability to sync exactly the contents of the
tagged release. This will make repeatable deployments faster for
downstream consumers.

The repo-store-source.yml play was removed because it hasn't been
used as the repo clone process is part of the main repo build
process.

DocImpact
UpgradeImpact
Change-Id: Icb0a4e3443491eb0242125ea59f8d5b50f89410b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2015-10-22 20:04:27 +00:00
Jesse Pretorius
e343d6e9fe Only wait for SSH if the container config has changed
This patch reduces the time it takes for the playbooks to execute when the
container configuration has not changed as the task to wait for a successful
ssh connection (after an initial delay) is not executed.

Change-Id: I52727d6422878c288884a70c466ccc98cd6fd0ff
2015-10-07 08:22:34 +01:00
Kevin Carter
2346e5ced4 Fixes log rotate issue
This change implements a change in the file name for each service
so that the log rotate files don't collide when running on a shared
host.

Change-Id: Ia42656e4568c43667d610aa8421d2fa25437e2aa
Closes-Bug: 1499799
2015-09-30 18:03:13 +00:00
Kevin Carter
4745e709f6 Removes over zealous arp cache flushing
This commit removes the use of the net-cache flushing from all
$service plays which ensures that the cache is not overly flushed
which could impact performance of services like neutron.

The role lxc-container-destroy role was removed because its not used
and if it were ever used it its use would result in the same
situation covered by this issue.

Additionally it was noted that on container restarts, the mac addresses
of the container interfaces change.  If *no* flushing is done at all,
this results in long run times whilst the arp entry for the container IP
times out.  Hence, we add in here a configuration option that causes a
gratuitous arp whenever an interface has it's mac set, and/or the link
comes up.  Because of the way the container veths work, we can't rely
on that happening on a linkm up event. So we forcefully set the mac
address in the post-up hook for the interface to force the issue of the
gratuitous arp.

Co-Authored-By: Evan Callicoat <diopter@gmail.com>
Co-Authored-By: Darren Birkett <darren.birkett@gmail.com>

Change-Id: I96800b2390ffbacb8341e5538545a3c3a4308cf3
Closes-Bug: 1497433
2015-09-29 13:21:29 +01:00
Jesse Pretorius
91fc87ae38 Additional retries for ssh wait check
This patch adds additional retries to the ssh wait check
tasks which were unintentionally omitted from
https://review.openstack.org/218572

Change-Id: Id8f7df5e283a9f61373f1bfb167a4c0bd098cc25
Closes-Bug: #1490142
2015-09-03 18:36:36 +00:00
Jesse Pretorius
a40cb58118 Wait for container ssh after apparmor profile update
This patch adds a wait for the container's sshd to be available
after the container's apparmor profile is updated. When the
profile is updated the container is restarted, so this wait is
essential to the success of the playbook's completion.

It also includes 3 retries which has been found to improve the
rate of success.

Due to an upstream change in behaviour with netaddr 0.7.16 we
need to pin the package to a lower version until Neutron is
adjusted and we bump the Neutron SHA.

Change-Id: I30575ee31929b0c9af6353b7255cdfb6cebd2104
Closes-Bug: #1490142
2015-09-02 09:21:55 +01:00
kevin
ffb701f8a3 Removed default lxc profile on container create
Having the lxc container create role drop the lxc-openstack apparmor
profile on all containers anytime its executed leads to the possibility
of the lxc container create task overwriting the running profile on a given
container. If this happens its likley to cause service interruption until the
correct profile is loaded for all containers its effected by the action.

To fix this issue the default "lxc-openstack" profile has been removed from the
lxc contianer create task and added to all plays that are known to be executed
within an lxc container. This will ensure that the profile is untouched on
subsequent runs of the lxc-container-create.yml play.

Change-Id: Ifa4640be60c18f1232cc7c8b281fb1dfc0119e56
Closes-Bug: 1487130
2015-08-25 13:15:45 +00:00
Stanley Karunditu
57c5f2c77e Add configurable ssh_delay
This patch adds a configurable delay time for retrying the
ssh connection when waiting for the containers to restart.

This is useful for environments where resources are constrained
and containers may take longer to restart.

Change-Id: I0383e34a273b93e1b2651460c853cf1ceba89029
Closes-Bug: #1476885
2015-08-22 11:43:02 +01:00
Stanley Karunditu
9e08d31fe2 Add regex check for ssh connection
This patch adds a check for the appropriate OpenSSH Daemon
reponse when waiting for the container to restart. This is
an optimisation over simply waiting for the TCP port.

Change-Id: Ie25af4f57bb98fb1d846d579b58b4d479b476675
Closes-Bug: #1476885
2015-08-22 11:18:52 +01:00
Kevin Carter
e34a775e04 Revert "changed container bind mounts to use abspath"
This reverts commit a6536b132c06231773525ba73398091b937b390f.

Change-Id: I52f5084c72ebedf9346dc4078fa18019bb7ea511
2015-06-23 17:38:44 +00:00
kevin
a6536b132c changed container bind mounts to use abspath
This change modifies the container create bind mounts to use the
absolute path that would be within the container instead of the
relitive path. This change is being done to ensure that there
are never issues with bind mounts as in newer versions of LXC and
the CGManager the absolute path is required.

Change-Id: I6af23c7ea0a7f905bdd587adde966a449402ed0a
Closes-Bug: #1462068
2015-06-16 19:53:44 -05:00
d34dh0r53
31da4f0331 Adds rsyslog-client tag to install plays
In order to ease the addition of external log receivers this adds an
rsyslog-client tag to the installation plays.  This allows us to run
openstack-ansible setup-everything.yml --tags rsyslog-client to add
additional logging configuration.

Change-Id: If002f67a626ff5fe3dc06d77c9295ede9369b3dc
Partially-Implements: blueprint master-kilofication
2015-04-16 08:10:46 +00:00
Kevin Carter
7c6e1260fe Updated ansible to use the latest stable release
This change was made to improve ansible stability and speed.
Additionally this change comes with the now upstream lxc-container
module which will allow us to drop our carried module. In dropping
the module the entry point was changed from `lxc-container:` to
`lxc_container:`. All of the entry points have been changed in
support of the new upstream module and the carried `lxc-container`
module has been removed.

Partially Implements Blueprint: master-kilofication
Partial-Bug: 1399373

Change-Id: I4709eb78f153afc213225ea973570efa2e873993
2015-04-08 08:29:53 +01:00
Kevin Carter
5b4eee1fc1 Adds rsyslog client role and enables it in all plays
This commit adds the rsyslog_client role to the general stack. This
change is part 3 of 3 role will allow rsyslog to server as a log
shipper within a given host / container. The role has been setup to
allow for logs to be shipped to multiple hosts and or other
providers, e.g. splunk, loggly, etc... All of the plays that need
to support logging have been modified to use the new rsyslog_client
role.

Roles added:
* rsyslog_client

Plays modified:
* playbooks/galera-install.yml
* playbooks/lxc-hosts-setup.yml
* playbooks/os-cinder-install.yml
* playbooks/os-glance-install.yml
* playbooks/os-heat-install.yml
* playbooks/os-horizon-install.yml
* playbooks/os-keystone-install.yml
* playbooks/os-neutron-install.yml
* playbooks/os-nova-install.yml
* playbooks/os-swift-install.yml
* playbooks/os-tempest-install.yml
* playbooks/rabbitmq-install.yml
* playbooks/repo-server.yml

DocImpact
Implements: blueprint rsyslog-update

Change-Id: I4028a58db3825adb8a5aa73dbaabbe353bb33046
2015-03-17 13:52:30 -05:00
Kevin Carter
8e6dbd01c9 Convert existing roles into galaxy roles
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.

Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
  simplistic approach. This change duplicates code within the roles but
  ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
  Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
  anyone who may want or need to dive into the JSON blob that is created.
  In the inventory a properties field is used for items that customize containers
  within the inventory.
* The environment map has been modified to support additional host groups to
  enable the seperation of infrastructure pieces. While the old infra_hosts group
  will still work this change allows for groups to be divided up into seperate
  chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
  variables extracted into the separate file
  etc/openstack_deploy/user_secrets.yml in order to allow seperate
  security settings on that file.

Items Excised:
* All of the roles have had the LXC logic removed from within them which
  should allow roles to be consumed outside of the `os-ansible-deployment`
  reference architecture.

Note:
* the directory rpc_deployment still exists and is presently pointed at plays
  containing a deprecation warning instructing the user to move to the standard
  playbooks directory.
* While all of the rackspace specific components and variables have been removed
  and or were refactored the repository still relies on an upstream mirror of
  Openstack built python files and container images. This upstream mirror is hosted
  at rackspace at "http://rpc-repo.rackspace.com" though this is
  not locked to and or tied to rackspace specific installations. This repository
  contains all of the needed code to create and/or clone your own mirror.

DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-18 10:56:25 +00:00