Commit Graph

40 Commits

Author SHA1 Message Date
Zuul
3d5f38f23c Merge "Add Bionic testing" 2018-05-14 20:59:11 +00:00
Jean-Philippe Evrard
2910c5ad60 Add Bionic testing
Now that bionic testing is added into the tests repos, we can
start testing it in the repo.

cgmanager isn't in bionic, and therefore is removed

The service module isn't in bionic, and therefore it's been renamed to
"systemd".

The apparmor setup we were doing was breaking the apparmor profiles
required. While this worked in xenial it breaks bionic. To fix this
we're just disabling the apparmor profiles instead of trying to to
augment them through block file changes.

Depends-On: https://review.openstack.org/#/c/566959/
Change-Id: Ie4bca80d0dba7b0da0b5829b91cd6d815894aeaa
Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com>
2018-05-14 21:04:09 +02:00
Zuul
bc1e87b611 Merge "Add mount options for better machinectl performance" 2018-05-12 19:55:20 +00:00
Mohammed Naser
030c348117 Setup /dev/random and /dev/urandom device in cache prep
In the cache preparation stage, there are certain libraries that
depend on the existance of /dev/random and /dev/urandom in order
to be able to function correctly, such as NSS in the latest CentOS
release (7.5)

This patch adds those nodes so that the libraries are able to use
them with no problems, allowing yum and rpm specifically to work
properly again.

Change-Id: Iaf6b9cb1435591f28289493f480a7fe46789c551
2018-05-10 18:02:28 +00:00
Kevin Carter
bf9a79d05e Add mount options for better machinectl performance
The machinectl default options, while functional, could be tuned for
better overall performance. This change adds several options which will
ensure container workloads are using the lest amount of storage with the
best possible performance.

For more information on the options being used see
 * https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs(5)#MOUNT_OPTIONS

All of the "machines" mount procedures have been moved into a unified
volume task file. This was done to ensure a consistent experience across
our supported distros. To ensure any new options are non-disruptive, the
mount handler has been changed to use "reload-or-restart" which will first
try to reload a mount instead of restarting it mounts.

Change-Id: Ia962fd4c5bb2a73ddd884d3bb3837c47b43d6903
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-05 17:43:40 +00:00
Logan V
9ad190a7c8 Fix locales on Ubuntu
The new Ubuntu base image we use does not ship with any locales
or locale configuration. A fix[1] attempted to remedy this by
copying the default locale configuration from the host, but it
is not a valid fix since the locale from the host is not present
or generated in the container cache. This causes things to break
on the system when valid locales are used, such as database
systems[2].

Instead, to prepare locales in Ubuntu[3], we should install the
locales package and provide a list of valid locales to prep
the base image. It is not necessary to copy /etc/default/locale
from the system. The first locale provided will be used as
the system's base locale by running 'update-locale' which
builds /etc/default/locale.

[1] e62de979cb
[2] http://paste.openstack.org/show/719241/
[3] https://www.thomas-krenn.com/en/wiki/Configure_Locales_in_Ubuntu#No_locale_set

Change-Id: Iaa5351777d7db464e8a897fdf33c0f440bfa601b
2018-04-14 17:43:48 -05:00
Logan V
1871fbec7b Fix lxc-system-manage ipv6 setup
When the lxc-dnsmasq service is restarted while ipv6 is active,
it will always fail to restart because the ip -6 addr add command
fails due to the address already existing on the interface.

ex.
Apr 01 22:39:48 lsn-mc1009 systemd[1]: Starting lxc dnsmasq service...
Apr 01 22:39:48 lsn-mc1009 lxc-system-manage[19134]: Creating LXC IPtables rules.
Apr 01 22:39:48 lsn-mc1009 lxc-system-manage[19134]: RTNETLINK answers: File exists
Apr 01 22:39:48 lsn-mc1009 systemd[1]: lxc-dnsmasq.service: Control process exited, code=exited status=2
Apr 01 22:39:48 lsn-mc1009 systemd[1]: Failed to start lxc dnsmasq service.
Apr 01 22:39:48 lsn-mc1009 systemd[1]: lxc-dnsmasq.service: Unit entered failed state.
Apr 01 22:39:48 lsn-mc1009 systemd[1]: lxc-dnsmasq.service: Failed with result 'exit-code'.
Apr 01 22:39:50 lsn-mc1009 systemd[1]: lxc-dnsmasq.service: Service hold-off time over, scheduling restart.

Change-Id: Ia3ebaf2125a00c4031f50bf03b60dd5659f9d660
2018-04-01 22:42:19 -05:00
Kevin Carter
a1a316f22e Correct image prep for minimal containers
Both CentOS and SUSE have pathing and services that are assumed to exist
from our previous use of the larger LXC images. This change also ensures
netbase is installed across our supported distros.

* Imports the "rpm-gpg" packages if the directory exists.
* unmasks the "sshd" sevice on SUSE if present and masked or disabled.
* Adds the (setup|netbase|netcfg) packages to the base image

Change-Id: Idc5caf2030c0e50dfeb84e0648bec08d7f50b6b8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-03-23 04:45:06 +00:00
Kevin Carter
e44df830ef Split the container and host variable files
The host and container image variable files have been split. This split
now gives deployers the ability to change or customize the container
image used on a given host.

Change-Id: I839bbcfff3f33dde144e9fb8d078fa1d97f8c410
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-03-21 23:53:12 +00:00
Kevin Carter
7e98da3d0f Convert lxc_hosts role to use simple download URL
For a very long time we've been parsing and using the lxc images as
provided by upstream lxc. While these images are functional there are by
no means optimal. In general they're quite a bit larger than they need
to be and contian a lot of little sharp edges that have cut us over
the years. This change removes all of the lxc image cache parsing and
meta-data linking and simply downloads the rootfs a given url. To
maintain compatibility with the legacy images a script has been created
to parse the image index and return the legacy image url.

The result of this change:

* Access to smaller more optimal base image which is well known by the
  corresponding communities.

* Deployers now have the ability to set and forget the download url for an
  internal image instead of having to create a cache infrastructure
  compatible with the lxc download template.

* Any rootfs tarball will work as an image.

* Fewer tasks are executed and less memory is consumed resulting in faster
  deployment times.

* The base cache has a uniform meta-data setup giving all container
  types the same access to config, devices, and templating.

Change-Id: I1775e775bbb7fe86bdffdd8296c2cff5ebc5bac8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-03-21 23:52:53 +00:00
Kevin Carter
0d8fa41d32 Move the image prep script into a template file
This change moves the image prep scripts out of a ser of variables and
into an actual template. This change will reduce our overall memory
footprint by simply rendering a template instead of injecting content
into a file using the copy module. The result will be faster time to
execution and more understandable output, especially when running in
debug.

Change-Id: Ic90fa7c8fdec8ffd844070ee78d30bd63a33a2a9
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-03-21 14:52:47 +00:00
Kevin Carter
44409262d2 Use local container meta-data
The current lxc meta-data process is one where we download an archive
from the upstream lxc images and store it locally on the host. While the
archive is small, this is a process that can break due to transient
networking issues and is an external dependency that we don't need.

The meta-data for the containers we build is all the same between
distros so it's easy to replicate and maintain as a local dependency.
This change creates a templates meta-data folder and stores our
required meta-data items within it. With this change we'll ensure
all containers are built with the same capabilities without requiring
access to an upstream repo and will improve the general speed of
deployment due to the task simplification and removal of an external
dependency.

Change-Id: I999d7068ce05645c477408fbd40556427c202a40
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-03-19 15:07:26 +00:00
Markos Chandras
f5c39d087e templates: lxc-system-manage: Send correct DNS over dhcp
We should set the correct DNS IP address via DHCP so containers can get
a valid entry. This fixes a problem where the DNS inherits the DNS
server from the host and the host uses the localhost IP in case it's
running a local DNS caching resolver such as dnsmasq.

Depends-On: Ied7632037f737c3f32c34dac70531065c54496c9
Change-Id: I14f8373897da28dea2ea03500c2be46c5b40d51c
2018-03-05 16:40:22 +00:00
Kevin Carter
bf143155f4
Cleanup the host prep role & remove redundant tasks
The LXC-Container-Create role now has the ability to setup all of the
network interfaces using systemd-networkd. Those changes give us a
uniform interface to consume when we create containers and free's the
roles from having to care about specific container interface config.
This change removes our now redundant tasks.

This also ensures "dbus" is available in the base contianer image
which was simply added for consistency.

Change-Id: I9278b1f73e1e0fdf98ab5fbe016a77aeb3f75be2
Depends-On: I5d3ddcfa11d575648a69a04f2fb30236c2c89da3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-02-28 08:34:15 -06:00
Major Hayden
a468089727
Allow NetworkManager to work (if present)
This patch adds tasks that check to see if NetworkManager is
installed and running. If it is, the tasks allow NetworkManager
to handle the `lxcbr0` interface.

In addition, the `NetworkManager-wait-online.service` will be
enabled to ensure that all services that depend on networking will
wait for network configuration to be completed.

Partial-Bug: 1738467
Change-Id: I415241daccf22f03826062eea18b3b36b2d9e53e
2018-01-08 09:22:51 -06:00
Kevin Carter
69ab10bee5
Add ExecStartPre command to lxc-dnsmasq service unit
When upgrading from an older deployment without the lxc-dnsmaq service
unit it's possible for the lxc-dnsmasq process to have been left in
a defunct state; this is especially true when upgrading as the
background process will have been started as a post-up command within the
lxcbr0 interface configuration. If the service is in a defunct state, or
running without a valid pid file, the systemd service unit will fail to
start.

This change ensures systemd will clean up all processes owned by the
lxc-dnsmaq user with a limited scope of processes matching a regex of
"^dnsmasq" which is only effectuve when the service is not managed by
systemd and in a defunct state.

Change-Id: If183f0b6dfbe0646384cf3bb3b89bc3901643c1e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-11-10 16:13:54 -06:00
Kevin Carter
53a6cce9ed
Use handlers to restart services and move dnsmasq to a unit file
These changes further optimise the lxc_host role so that it's using more
of the built in modules and making better use of handlers.

Moving the dnsmasq process to a unit file gives operators the ability to
restart the dnsmasq process if there's an issue with the service. It
also ensures the service stays running as systemd will take better care
of the service by isolating it within a specific cgroup, ensuring good
reporting and memory management, and providing the ability to recover
from failures in an automated way.

Closes-Bug: #1518485
Change-Id: I42d0caa3b12e70a3601c30051eefc067e81a71bb
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-11-01 15:19:22 -05:00
Daniel Speichert
e43304f3df Stop trying to set MTU on LXC bridge interface
This setting is invalid on the bridge and prevents it
from coming up. The bridge uses the lowest MTU used by
all of its connected ports automatically, so any MTU
configured for LXC containers will be used by the bridge.

Change-Id: I8ba8c43492493c4de46469903f0567b6ca7b509d
Closes-Bug: 1724337
2017-10-19 15:58:51 -04:00
Andy McCrae
eee919ad21 Fix lxc_net configuration for CentOS/SUSE
For CentOS/Red Hat and SUSE the network post-up/post-down scripts are
configured after the bridge has been brought up, and the handlers have
been flushed.

We need to configure the post-up and post-down scripts before the
restart bridge handler is flushed, so that dnsmasq is configured and
running before we attempt to install packages into the container.

Change-Id: Ifdb52624ed792665c858b3cdd4eec4b6aa365b1e
2017-09-12 15:39:09 -06:00
Major Hayden
2ffcc79451
Prevent dnsmasq from reading hosts file
OpenStack-Ansible configures /etc/hosts with the management IP
addresses of containers, but dnsmasq gets confused when it sees those
host names associated with a 10.X address on the lxc bridge. It
causes lots of errors in the log files like these:

  not giving name hydrogen_rsyslog_container-307df194 to the DHCP
  lease of 10.0.3.196 because the name exists in /etc/hosts with
  address 172.29.239.71

This patch adds the --no-hosts option for dnsmasq so that it will
stop reading /etc/hosts and filling up log files.

Closes-Bug: 1668949
Change-Id: I7aa2f0081d7d79ab42fcfc28e3ed6839a4f66c8a
2017-08-17 13:47:23 -05:00
Major Hayden
de1b45553e Download LXC image with async via aria2
This patch adds an async task to download the LXC image using aria2
with retries and read timeouts.

Closes-Bug: 1709329
Change-Id: Ib9ec6195dcb7e0e4b18b8526f030e6738f9953e8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-15 12:34:26 +00:00
Kevin Carter
0e6bcf2a1b
remove the use of iteritems
iteritems is a py2 callable and will break when using py3.

Change-Id: I2afbcf273d23a326b81e51620929a512ed17fc22
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2017-08-02 15:06:50 -05:00
Markos Chandras
583337b69b Fix container bridge networking on SUSE
The GATEWAY attribute does not exist on SUSE so we need to add the
default gateway to an ifroute file. Moreover, we drop the DEVICE
attribute which also doesn't exist on SUSE ifcfg files and we add
STARTMODE='auto' to instruct wicked to automatically configure the
interface on every boot. Finally we switch STP to 'off' since there
is only a single bridge accessible from containers.

Change-Id: I9c09e2882614f89cb944a623e7b97298d4bec541
2017-05-08 11:24:12 +01:00
Markos Chandras
09406de86b Add SUSE support
Add support for SUSE based distributions. We also update the bindep.txt,
run_tests.sh and Vagrantfile files from the openstack-ansible-tests
repository except that we use Leap 42.1 because Leap 42.2 does not work
as expected with the currently released bindep (2.3.0).

Change-Id: I5fb94a7cedf9d28816184e3eadd88e42f93295c2
2017-04-25 14:56:33 +01:00
Markos Chandras
cd67263120 tasks: Set systemd installation prefix
The systemd installation prefix is distro specific so add a new
systemd_utils_prefix variable in the distro files instead of using
a hardcoded value.

Change-Id: Idccb404696e1cf3b8d56f62782dc8afa86996517
2017-04-25 14:56:33 +01:00
Logan V
89625e66b0 Configure proxy settings for machined
The machined pull does not utilize the lxc_cache_environment
variable to configure its environment, so it will not use proxies.

This fixes machined to properly configure its environment so pulls
can operate through proxies.

Based on https://wbl.krm.io/howto-proxify-systemd-machined/

Change-Id: I17c452a8ba67561435737ee9d672219a2e306489
2017-04-22 00:57:27 +00:00
Jenkins
0b5fd2fc49 Merge "Add lxc_net_manage_iptables variable" 2017-03-24 16:03:48 +00:00
Ravi Kumar Boyapati
fff13e53d7 Add lxc_net_manage_iptables variable
Added lxc_net_manage_iptables variable. Setting this variable avoids
duplcaiting the existing rules.

Change-Id: I76800d23929bd2a6f656a20095b7d2352a4757fa
2017-03-23 16:43:20 -04:00
Ravi Kumar Boyapati
4f900f1690 Fix the lock type variable
Fixed the lock variable to force iptables to acquire the lock
before adding/deleting rules.

Change-Id: If2307681db056302c9a677961194d9dde87de137
2017-03-22 23:07:45 -04:00
Jean-Philippe Evrard
fb92fc666c Remove remaining files from Trusty support removal
The upstart script is not used anymore we should not carry it.

Change-Id: I73671350db40444d3cb82e9e1a70a906e809f07f
2016-12-21 08:10:44 +00:00
Logan V
7568621597 Add support for IPv6 LXC network
Change-Id: If974bd8c3f8d6c5b5af14a3737ec5d89726cbe0f
2016-10-19 07:41:32 -05:00
Marc Gariepy
cf65c6736b Remove requiretty for sudo on centos base image.
In order to be able to use become: yes to execute command with ansible
the requiretty option needs to be disabled on centos

Change-Id: Iab2408267fdcb1c19c3c560d2f86181a6fc180e0
2016-09-15 20:18:56 +00:00
Marc Gariepy
812e72b7b2 Fix path for lxc-net config on distro.
Introducing system_config_dir to configure where the lxc-net file is
located on centos or ubuntu.

Change-Id: I3e2bbfd81f17b8a697ed9d7cad81c89b2b48ba9f
2016-09-01 10:23:55 -04:00
Kevin Carter
f5542103b3
Changed for lxc-host setup/build for multi-distro
This change updates the lxc-host setup role to build the lxc cache using the
download template based on default images found here:[0]. These images are
upsteam builds from the greater LXC/D community.

This update adds support for Ubuntu 14.04, 16.04 and RHEL/CentOS 7 container
types and the cache will be generated from the host Operating system.

[0] - https://images.linuxcontainers.org/

Change-Id: Ie13be2322d28178760481c59805101d6aeef4f36
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-05-03 08:49:54 -05:00
tengqm
aae22d17a3 Minor tweak to the lxc-system-manage template
This patch tweaks the lxc-system-manage template with typo
corrections.

Change-Id: I8e03cd823aaf7a4217afc3f7ae18d2a992b689bb
2016-04-09 10:47:05 -04:00
Jimmy McCrory
158d035b92 Make corrections to LXC bridge template file
This change adjusts a few of the modifications made to the
lxc-net-bridge.cfg.j2 template file in change
I3c8225124a5f18db81259e1d52d0168ef52c3c17.

The minus signs have been removed from if and endif blocks so that
whitespace is kept intact between sections. The ordering of post-up and
post-down commands has also been changed so that iptables rules are
created before the dnsmasq service is started, as they were previously.

The default value of lxc_net_gateway has also been changed to null so
that it's evaluated as expected. Its current value, none, is evaluated
as a string.

A test has been added to compare the contents of the deployed lxc bridge
interface file with its expected contents.

Change-Id: I39d7b3f40de6ac691550c11d71bb6a182b3452f4
2016-03-05 11:41:31 -08:00
Kevin Carter
fe999d1715 Resolve bad assumptions about the base OS
The change moves several tasks around and adds packages to the install
process which were previously assumed to be present on the base OS.

This also updates the lxc-net-bridge template to be more configurable
to address issues where the base OS is more minimal than previously
expected.

Change-Id: I3c8225124a5f18db81259e1d52d0168ef52c3c17
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-03-04 21:04:11 +00:00
Jesse Pretorius
f21bb596ab Set container apt sources to use a configured list of components
re: http://lists.openstack.org/pipermail/openstack-dev/2016-February/086000.html

Ubuntu has 4 different 'components' - main, universe, multiverse and
restricted:
 - Main: Officially supported software.
 - Restricted: Supported software that is not available under a completely
               free license.
 - Universe: Community maintained software, i.e. not officially supported
             software.
 - Multiverse: Software that is not free.

Practically speaking there should be nothing particularly useful to
OpenStack-Ansible in Restricted or Multiverse - it's mostly software for
desktop users.

This patch introduces a new variable 'lxc_container_template_apt_components'
which is a list of the components to configure in the apt sources list. The
default list does not include the unnecessary components.

Change-Id: I0eef6454d273f5ba1977a68151fbf6103ff3ed8d
2016-02-12 14:32:01 +00:00
Jean-Philippe Evrard
513244b56b Added to ability to set mtu for lxcbr0
The lxc_host role overrides the configuration
done by the system administrator on the host.

If mtu was defined in /etc/lxc/ and in the
system network interfaces, then these configurations
will be overriden.

This commit should allow the deployer to set the
mtu for its lxcbr0 network, by setting the variable
lxc_net_mtu.

Fixes-Bug: #1518303

Change-Id: I42ab3d0a2c20ae94335de195cdb14579d9f59f5f
2015-12-02 17:13:24 +01:00
Kevin Carter
eb9f3d858b IRR for lxc_host
The change moves the role out from the main repo lxc_host
repository and into its own standalone repository.

Items within this change:
  * The role has been updated to ensure it runs standalone.
  * Tests added to the role within tox.
  * Functional tests added to the role that can either be run
    via the run_tests.sh script or using tox.
  * dev requirements have been updated for testing usecases.
  * Docs added to both the README.rst file as well as the docs
    folder.

Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2015-11-03 04:22:57 -06:00